logging - Aggregating Hadoop job counters across multiple jobs -


hadoop: (ver - 1.2.1), (1 + 8 node cluster)

my use case is, trying time taken execute specific pig script , how time being spent mapreduce point of view. need run pig script multiple times(say 100) average time. i've enabled pig.udf.profile gives me time spent on each udf function mapreduce counters. interested in other latency,memory metrics reported each job (cpu time,heap useage). can see these counters jobtracker web ui( host:50030/jobdetails.jsp?jobid=blah). now, question is, there way aggregate these counters across jobs. or, how build table looks like

                        run1   run2   run3 ... cpu time              |      |      | redcr wait            |      |      | udfcntr1(approx_us)   |      |      | udfcntr2(approx_invc) |      |      | countery(approx_us)   |      |      |

each run different job far hadoop concerned. after grep'ing through log folder, figured out counters in history/done/.. folders. there existing technique combine results, or doomed write own parser goes through each log file. thought use case common enough existing solution - pointers helpful.

thanks.

you have couple of options, , apologize in advance none of them particularlly appealing.

implement ppnl

the pigprogressnotificationlistener java interface exposing events occur on course of pig job arbitrary clients. if implement interface , attach instance of class can grab hadoop counters (and many other m/r related metrics) , store them away later usage. note requires reasonable understand of pig internals, though not expert level of understanding.


use system gathers metrics

right options limited ambrose open sourced twitter, , lipstick open sourced netflix. not know sure if ambrose collects hadoop counters, extended do. lipstick collect hadoop counters is. either of these analyze counters varying level of difficulty, depending on how configured them store data.


parse log files

sounds you've thought going down route couple of reasons:

  1. all other options require know quite internals of project, pig, lipstick, or ambrose. if planning dive 1 of these other reasons go it, otherwise it's big investment single usecase.
  2. all other options limited pig jobs. aren't going data if run vanilla map/reduce job or start working other tooling hive, cascading, etc. if build log parsing tooling converts data known format gets processed should able reuse code (except actual parsing) hadoop tools.
  3. if data based on logs, , archive logs, can backfill if 6 months down road you're interested in data point hadn't collected before. isn't possible other approaches.
  4. it's established pattern works. can't point lot of online resources discussing people have taken approach, have met quite few people independently took approach , happy it.

Comments

Post a Comment

Popular posts from this blog

c++ - CryptStringToBinary API behavior -

i seeing strange behavior cryptstringtobinary cryptography api. please see below code (config: x64 debug): #include "stdafx.h" #include <windows.h> #include <strsafe.h> #include <iostream> #include <exception> void main() { dword dwskip; dword dwflags; dword dwdatalen; //lpcwstr pszinput = l"jaaaaaecaaadzgaaakqaagdnnl1l56bwgfjdgr3rpxtqqqn6daw3usv2emkjym4t"; //this works fine lpcwstr pszinput = l"mytest"; //doesnt work, api returns false,error code 0x0000000d // determine size of byte array , allocate memory. if(! cryptstringtobinary( pszinput, _tcslen( pszinput ) + 1, crypt_string_base64, null, &dwdatalen, &dwskip, &dwflags ) ) { dword dw = getlasterror(); //0x0000000d: data invalid throw std::exception( "error computing byte length." ); } byte *pbytebyte = null; try { ...
Read more

java.util.scanner - How to read and add only numbers to array from a text file -

i'm learning java , have question regarding reading file want read numbers file contains strings well. here example of file: 66.56 "3 java 3-43 5-42 2.1 1 and here coding: public class test { public static void main (string [] args){ if (0 < args.length) { file x = new file(args[0]); try{ scanner in = new scanner( new fileinputstream(x)); arraylist<double> test = new arraylist<>(); while(in.hasnext()){ if(in.hasnextdouble()){ double f=in.nextdouble(); test.add(f);} else {in.next();} } catch(ioexception e) { system.err.println("exception during reading: " + e); } } my problem add 66.56,2.1 , 1 doesn't add 3 after "3 or ignores 3-43 , 5-42 can tell me how skip strings , add doubles here? thanks all said 3 ie; "3, 3-43 , 4-42 strings either u read string , split , check number @ " , - or put in space between characters , integers. jvm after compilation tre...
Read more

iphone - Three second countdown in cocos2d -

i trying have 3 second countdown display when level beat in game. did: -(void) gamesegmentbeat { [self pauseschedulerandactions]; // overlay game opaque background ccsprite *opaquebg = [ccsprite spritewithfile:@"background1.png"]; opaquebg.position = screencenter; [self addchild:opaquebg z:10000]; // these 3 labels 3 seconds cclabelttf *three = [cclabelttf labelwithstring:@"3" fontname:@"helvetica" fontsize:100]; three.position = ccp(screencenter.x, screencenter.y); cclabelttf *two = [cclabelttf labelwithstring:@"2" fontname:@"helvetica" fontsize:100]; two.position = ccp(screencenter.x, screencenter.y); cclabelttf *one = [cclabelttf labelwithstring:@"1" fontname:@"helvetica" fontsize:100]; one.position = ccp(screencenter.x, screencenter.y); // secondspast specified in update method secondspast = 0; if (secondspast == 1) { [self addchild:thr...
Read more