Page 1 of 1

problem with search output

Posted: Tue 21. Sep 2010, 18:56
by helresa
Hi all

I´ve got a little problem with the output from search, and characters that have an accent (The site is in Spanish so they are a few) are not rendering corectly in the search output
Image

php 5
phpwcms 1.4.5 (2010/02/08, r398).
lang utf8

Re: problem with search output

Posted: Tue 21. Sep 2010, 19:01
by update
Some where there will be no utf_8 I bet!
Are all used templates saved in utf-8?
What is the database talking - have a look at the tables...

Re: problem with search output

Posted: Tue 21. Sep 2010, 19:05
by helresa
all the normal pages output correctly

The template is the same

the database is running latin1_swedish_ci which is the default...

Re: problem with search output

Posted: Tue 21. Sep 2010, 19:13
by update
helresa wrote:the database is running latin1_swedish_ci which is the default...
But this isn't utf, is it? What are the tables looking like?
What is the content of your conf.inc.php concerning collation and so on (when posting, please don't post PWs)

Re: problem with search output

Posted: Tue 21. Sep 2010, 19:25
by helresa
$phpwcms['db_pers'] = 1;
$phpwcms['db_charset'] = 'latin1';
$phpwcms['db_collation'] = 'latin1_swedish_ci';
$phpwcms['db_version'] = 0;

$phpwcms['charset'] = 'utf8';

and in the database the article summary

Code: Select all

<p>&ldquo;City Hope&rdquo; de Javier Arcenillas, ideada como una panor&aacute;mica visual-fotogr&aacute;fica confinada en las ciudades vertedero de Am&eacute;rica Latina, bien podr&iacute;a entenderse como un gran relato universal narrado desde los confines que dividen la riqueza y la pobreza, el despilfarro y la dignidad personificada en los 

Re: problem with search output

Posted: Tue 21. Sep 2010, 19:31
by update
For me this seems to be out of order a little bit. Hope that flip-flop is coming around again soon. Probably he will know better what to do....
Sorry for being of no further help :(

Re: problem with search output

Posted: Sat 25. Sep 2010, 14:12
by flip-flop
Your problem is the non-matching character sets from the db and the page.
In the future you will always get problems with the combination latin1 and utf8.

Knut

Re: problem with search output

Posted: Sat 2. Oct 2010, 09:18
by marcus@localhorst
howdy,
today I ran into the same problems.

Long Version:
Everything was set to utf8
Seachresulttext was mangled if CKEditor with HTML entities was used.
entities conversation is made in cleanUpSpecialHtmlEntities() in general.inc.php
it's related to a charset table/var to search and replace entities with the right utf-8 char called $GLOBALS['SPECIAL_ENTITIES_TABLES']
on top of general.inc.php the file

Code: Select all

require_once (PHPWCMS_ROOT.'/include/inc_lib/lib.php_special_entities.php');
is included.
This file is encoded as ANSI text.
There is another file called /include/inc_lib/lib.php_special_entities.UTF8.php
this file is encoded as utf-8 text.

if you change the inclusion to

Code: Select all

require_once (PHPWCMS_ROOT.'/include/inc_lib/lib.php_special_entities.UTF8.php');
it will work :)

So I don't know if this will have any side effects, but for now it works.
And I don't know if there should be a check against $phpwcms['charset'] to include the right file?

I will send a Report to phpwcms google code...

cheers

Re: problem with search output

Posted: Sat 2. Oct 2010, 09:40
by Oliver Georgi
Change the first lines of include/inc_lib/general.inc.php to:

Code: Select all

if(PHPWCMS_CHARSET == 'utf-8') {
	require_once (PHPWCMS_ROOT.'/include/inc_lib/lib.php_special_entities.UTF8.php');
} else {
	require_once (PHPWCMS_ROOT.'/include/inc_lib/lib.php_special_entities.php');
}
require_once (PHPWCMS_ROOT.'/include/inc_lib/charset_helper.inc.php');
…
In a future release I will reject support for non UTF-8 charsets from phpwcms. Makes no sense any longer. But will need some more things to be done because it is necessary then to have an automatic upgrade process making charset conversion on the fly.

Re: problem with search output

Posted: Fri 8. Oct 2010, 16:38
by juergen
Oliver Georgi wrote:Change the first lines of include/inc_lib/general.inc.php to:
:shock: do while stumbling over the old rules .... ;)