hadoop - Datatype mismatch causing comparison failure? Python UDF in Pig -


i'm having trouble python udf use in pig scripts. believe problem assumed input deltas in format it's not in, i'm not sure how fix (python n00b).

note: on cloudera (cdh4.3) distro of hadoop v.2.0.0, pig v.0.11.0, python 2.4.3.

import org.apache.pig.impl.logicallayer.schema.schemautil schemautil  @outputschema("adj:float") def cumrelfreqadj(deltas):      # create bins of increment 0.01     = [i*-0.01 in range(100)]     = a[1:len(a)]     b = [i*0.01 in range(101)]     a.extend(b)     a.sort()     bins =      # build cumulative relative frequency distribution     cumfreq = [0]*200     delta in deltas:         bin in range(len(bins)):             if delta <= bins[bin]:                 cumfreq[bin] += 1      cumrelfreq = [float(cumfreq[i]) / max(cumfreq) in range(len(cumfreq))]      crf = zip(bins, cumrelfreq)      relfreq in crf[:]:         if relfreq[1] > 0.11:    # 10%ile             adj = relfreq[0] + 0.05             break      return adj

do need convert input list first?

answered own question. input pig bag of tuples. in case each tuple has 1 element, e.g.: {(-0.01), (-0.03), (0.00001), (-0.2383), (0.158)}.

so in order compare float-type element list bins, need insert like:

delta = list(delta)[0]

between lines 16 & 17 above, pull out float-typed data element content of tuple. comparison on line 18 work.


Comments

Post a Comment

Popular posts from this blog

java.util.scanner - How to read and add only numbers to array from a text file -

i'm learning java , have question regarding reading file want read numbers file contains strings well. here example of file: 66.56 "3 java 3-43 5-42 2.1 1 and here coding: public class test { public static void main (string [] args){ if (0 < args.length) { file x = new file(args[0]); try{ scanner in = new scanner( new fileinputstream(x)); arraylist<double> test = new arraylist<>(); while(in.hasnext()){ if(in.hasnextdouble()){ double f=in.nextdouble(); test.add(f);} else {in.next();} } catch(ioexception e) { system.err.println("exception during reading: " + e); } } my problem add 66.56,2.1 , 1 doesn't add 3 after "3 or ignores 3-43 , 5-42 can tell me how skip strings , add doubles here? thanks all said 3 ie; "3, 3-43 , 4-42 strings either u read string , split , check number @ " , - or put in space between characters , integers. jvm after compilation tre...
Read more

rewrite - Trouble with Wordpress multiple custom querystrings -

i'm working on adding multiple querystring , doesn't seem work. i've found codes seems work others might i'm missing something. i want like: http://mydomain.com/board/?getyear=2013&getsometext2=sometext to http://mydomain.com/board/2013/sometext/ i added code below functions.php file. doesn't seem work should though. this works. echoing querystring gets both variable without problem. http://mydomain.com/board/2013/sometext/ but doesn't. tested querystring vis isset() seems nothing being set/get http://mydomain.com/board/2013/ function add_query_vars($vars) { $vars[] = "getyear"; $vars[] = "getsometext2"; return $vars; } add_filter('query_vars', 'add_query_vars'); function add_rewrite_rules($arules) { $anewrules = array('board/([^/]+)/([^/]+)/?$' => 'index.php?pagename=board&getyear=$matches[1]&getsometext2=$matches[2]'); $arules = $a...
Read more