encoding - unicode:characters_to_list seems doesn't work for utf8 list -


i trying convert utf-8 string unicode (code point) list erlang library "unicode. input data string "АБВ" (russian string, correct unicode representation [1040,1041,1042]), encoded in utf-8. when running following code:

1> unicode:characters_to_list(<<208,144,208,145,208,146>>,utf8). [1040,1041,1042]

it returns correct value, following:

2> unicode:characters_to_list([208,144,208,145,208,146],utf8).   [208,144,208,145,208,146]

does not. why happens? read in specification, input data either binary or list of chars, so, me, doing right.

the signature of function unicode:characters_to_list(data, inencoding), expects data either binary containing string encoded in inencoding encoding or possibly deep list of characters (code points) , binaries in inencoding encoding. returns list of unicode characters. characters in erlang integers.

when call unicode:characters_to_list(<<208,144,208,145,208,146>>, utf8) or unicode:characters_to_list([1040,1041,1042], utf8) correctly decodes unicode string (yes, second noop long data list of integers). when call unicode:characters_to_list([208,144,208,145,208,146], utf8) erlang thinks pass list of 6 characters in utf8 encoding, since it's unicode output same.

there no byte type in erlang, assume unicode:characters_to_list/2 accept list of bytes , behave correctly.

to sum up. there 2 usual ways represent string in erlang, bitstrings , lists of characters. unicode:characters_to_list(data, inencoding) takes string data in 1 of these representations (or combination of them) in inencoding encoding , converts list of unicode codepoints.

if have list [208,144,208,145,208,146] in example can convert binary using erlang:list_to_binary/1 , pass unicode:characters_to_list/2, i.e.

1> unicode:characters_to_list(list_to_binary([208,144,208,145,208,146]), utf8). [1040,1041,1042]

unicode module supports unicode , latin-1. thus, (since function expects codepoints of unicode or latin-1) characters_to_list not need list in case of flat list of codepoints. however, list may deep (unicode:characters_to_list([[1040],1041,<<1042/utf8>>]).). reason support list datatype data argument.


Comments

Post a Comment

Popular posts from this blog

java.util.scanner - How to read and add only numbers to array from a text file -

i'm learning java , have question regarding reading file want read numbers file contains strings well. here example of file: 66.56 "3 java 3-43 5-42 2.1 1 and here coding: public class test { public static void main (string [] args){ if (0 < args.length) { file x = new file(args[0]); try{ scanner in = new scanner( new fileinputstream(x)); arraylist<double> test = new arraylist<>(); while(in.hasnext()){ if(in.hasnextdouble()){ double f=in.nextdouble(); test.add(f);} else {in.next();} } catch(ioexception e) { system.err.println("exception during reading: " + e); } } my problem add 66.56,2.1 , 1 doesn't add 3 after "3 or ignores 3-43 , 5-42 can tell me how skip strings , add doubles here? thanks all said 3 ie; "3, 3-43 , 4-42 strings either u read string , split , check number @ " , - or put in space between characters , integers. jvm after compilation tre
Read more

rewrite - Trouble with Wordpress multiple custom querystrings -

i'm working on adding multiple querystring , doesn't seem work. i've found codes seems work others might i'm missing something. i want like: http://mydomain.com/board/?getyear=2013&getsometext2=sometext to http://mydomain.com/board/2013/sometext/ i added code below functions.php file. doesn't seem work should though. this works. echoing querystring gets both variable without problem. http://mydomain.com/board/2013/sometext/ but doesn't. tested querystring vis isset() seems nothing being set/get http://mydomain.com/board/2013/ function add_query_vars($vars) { $vars[] = "getyear"; $vars[] = "getsometext2"; return $vars; } add_filter('query_vars', 'add_query_vars'); function add_rewrite_rules($arules) { $anewrules = array('board/([^/]+)/([^/]+)/?$' => 'index.php?pagename=board&getyear=$matches[1]&getsometext2=$matches[2]'); $arules = $a
Read more

php - Accessing static methods using newly created $obj or using class Name -

see class definition below: i using 5.3.9 version of php class a{ static function ab(){ echo "static function ab<br>"; } public function xy(){ echo "public function xy<br>"; } } $obj = new a(); $obj->ab(); a::ab(); both functions call give same output without error static function ab static function ab how possible static method can called class object? because static method calls using class name only?! now difference between accessing these 2 ways call static method ? referring php.net website declaring class properties or methods static makes them accessible without needing instantiation of class. property declared static can not accessed instantiated class object (though static method can). a big difference is because static methods callable without instance of object created, pseudo-variable $this not available inside method declared static. refer page php.net/manual/en/
Read more