unix - CSV - remove rows in which any column is empty -


i'm playing titanic data set kaggle. i'd remove rows train.csv have empty column (i know isn't best way deal missing data, question interesting me regardless).

i'd unix-type way (using awk, sed, or grep), because i'm trying better @ tools, i'm not sure start.

example of data:

passengerid,survived,pclass,name,sex,age,sibsp,parch,ticket,fare,cabin,embarked 1,0,3,"braund, mr. owen harris",male,22,1,0,a/5 21171,7.25,,s 2,1,1,"cumings, mrs. john bradley (florence briggs thayer)",female,38,1,0,pc 17599,71.2833,c85,c 3,1,3,"heikkinen, miss. laina",female,26,0,0,ston/o2. 3101282,7.925,,s

in second row, cabin empty, want remove file.

note fourth column contains commas, column contained in double quotes.

aside:

i'd know how specific columns, can ask separate question if answer question doesn't me answer one.

i stick language has parser because commas inside double quotes can problematic. , easier extend compare specific columns. here example. extracts number of fields header , compare number each line decide if print or not:

import sys  import csv   open(sys.argv[1], 'r', newline='') csvfile:     csvreader = csv.reader(csvfile)     csvwriter = csv.writer(sys.stdout)     row = next(csvreader)     fields = len(row)     csvwriter.writerow(row)     row in csvreader:         l = len(list(filter(str.strip, row)))         if l < fields: continue         csvwriter.writerow(row)

assuming code inside file name script.py, run like:

python script.py infile

that yields:

passengerid,survived,pclass,name,sex,age,sibsp,parch,ticket,fare,cabin,embarked 2,1,1,"cumings, mrs. john bradley (florence briggs thayer)",female,38,1,0,pc 17599,71.2833,c85,c

Comments

Post a Comment

Popular posts from this blog

java.util.scanner - How to read and add only numbers to array from a text file -

i'm learning java , have question regarding reading file want read numbers file contains strings well. here example of file: 66.56 "3 java 3-43 5-42 2.1 1 and here coding: public class test { public static void main (string [] args){ if (0 < args.length) { file x = new file(args[0]); try{ scanner in = new scanner( new fileinputstream(x)); arraylist<double> test = new arraylist<>(); while(in.hasnext()){ if(in.hasnextdouble()){ double f=in.nextdouble(); test.add(f);} else {in.next();} } catch(ioexception e) { system.err.println("exception during reading: " + e); } } my problem add 66.56,2.1 , 1 doesn't add 3 after "3 or ignores 3-43 , 5-42 can tell me how skip strings , add doubles here? thanks all said 3 ie; "3, 3-43 , 4-42 strings either u read string , split , check number @ " , - or put in space between characters , integers. jvm after compilation tre...
Read more

rewrite - Trouble with Wordpress multiple custom querystrings -

i'm working on adding multiple querystring , doesn't seem work. i've found codes seems work others might i'm missing something. i want like: http://mydomain.com/board/?getyear=2013&getsometext2=sometext to http://mydomain.com/board/2013/sometext/ i added code below functions.php file. doesn't seem work should though. this works. echoing querystring gets both variable without problem. http://mydomain.com/board/2013/sometext/ but doesn't. tested querystring vis isset() seems nothing being set/get http://mydomain.com/board/2013/ function add_query_vars($vars) { $vars[] = "getyear"; $vars[] = "getsometext2"; return $vars; } add_filter('query_vars', 'add_query_vars'); function add_rewrite_rules($arules) { $anewrules = array('board/([^/]+)/([^/]+)/?$' => 'index.php?pagename=board&getyear=$matches[1]&getsometext2=$matches[2]'); $arules = $a...
Read more