[BlueOnyx:12974] Re: Problem: user names disappears at Japanesecontrol page

[Michael Stauber]

Hi Eiji,

> May I ask you about the status of the fixed without creating new many
> problems ?

Thanks for reminding me. Since we last spoke I've been working day and
night on the new BlueOnyx GUI and didn't really have the time to fix
this. I looked into the nature of this problem, as I had a similar
Japanese related problem in the new GUI, too.

> If the fixed is not find yet,  can I use a local solution as temporary ?
> 
>   At   /usr/sausalito/ui/libPhp/uifc/FormFieldBuilder.php,    line about 
> 499.
>    I  changed  the statrment
>       $out = htmlspecialchars(stripslashes(trim($out)), ENT_QUOTES,'UTF-8');
>    to
>       $out = htmlspecialchars(stripslashes(trim($out)), ENT_QUOTES,'EUC');
> 
> My testing including English and Japanese are all O.K.

I can see how that might fix the display problem in that particular
usage case. But it has the chance of creating other problems. Not
necessarily on this page, but others:

When you display characters in UTF-8 in a GUI form and save the changes,
the data is saved to CODB in UTF-8 format as well. Same for EUC-JP: If
you save EUC-JP encoded form data, the GUI will save it to CODB in
EUC-JP format as well.

However, if you then view this saved EUC-JP encoded data again in a page
that has UTF-8 set, you see garbage. And vice versa.

Switching the entire encoding to EUC might cause that data that is
already stored into CODB in other formats might not be presented
correctly again.

The Class FormFieldBuilder is used in many places throughout the old
GUI. So I suspect your change will cause display problems elsewhere. I
could be wrong on this, but I don't want to risk this kind of potential
problem.

The proper work around (like the one I use in the new BlueOnyx) is this:

When we poll data from CODB for display, we check the encoding of the
data before it is presented. If the encoding of the data does not match
the currently used encoding for display (either UTF-8 or EUC-JP), then
the encoded data is re-encoded into the currently used encoding.

In the new GUI that function looks like this:

function bx_charsetsafe($string) {

  $CI =& get_instance();
  $locale = $CI->input->cookie('locale');

  if ($locale == 'ja_JP') {
      $charset = 'EUC-JP';
  }
  else {
      $charset = 'UTF-8';
  }
  $oldEncoding = mb_detect_encoding($string, "EUC-JP, UTF-8");
  if ($oldEncoding != $charset) {
    if ($oldEncoding == 'EUC-JP') {
	$encConv = new EncodingConv($string, 'japanese', 'EUC-JP');
	$string = $encConv->toUTF8();
    }
    else {
	$encConv = new EncodingConv($string, 'japanese', 'UTF-8');
	$string = $encConv->toEUC();
    }
  }
  return $string;
}

It is called like this from within a GUI page:

// Make the users fullName safe for all charsets:
$user['fullName'] = bx_charsetsafe($user['fullName']);

This function (as is) will not work in the old BlueOnyx, because it
makes reference to CodeIgniter's CI variable and CI's cookie function.
Likewise, the currently used locale is not stored in a cookie in the old
GUI, but in a PHP $GLOBAL variable that's actually been deprecated in PHP.

I'll try to wiggle a modified version of this function into
FormFieldBuilder for the old BlueOnyx.


-- 
With best regards

Michael Stauber