bash - Parse two files and merge lines if time-stamp matches -

i have 2 data files of following format:

file1:

date,time,data1,data2,data3 date,time,data1,data2,data3 date,time,data1,data2,data3 

file2:

date,time,data4 date,time,data4 

what want merge lines in these 2 files in such way if date , time match output should be:

date,time,data1,data2,data3,data4 

in case the time-stamp 1 file not match in other file can ignore line.

i'm doing using combination of awk , join. wonder if effective way or not.

current working implementation this:

 awk 'fs="," {print $1"&"$2 ","$3","$4","$5}' file1 > temp1  awk 'fs="," {print $1"&"$2 ","$3}' file2 > temp2  join -t',' -j1 1 -o 1.1,1.2,1.3,2.2,2.3 temp1 temp2 > temp3  awk 'fs="&" {print $1","$2"}' temp3 > output 

the way i'd (assuming dates , times have precisely matching format) be:

proc tidyuptimestamp {date time} {     # if want parse/tidy timestamp, here     return $date,$time }  # schlurp data array ease of access. few million lines set f [open "file2.csv"] foreach line [split [read $f] "\n"] {     lassign [split $line ","] date time data4     set map([tidyuptimestamp $date $time]) $data4 } close $f  # assuming file1.csv longer file2.csv set fin [open "file1.csv"] set fout [open "file1.processed.csv" w] while {[gets $fin line] >= 0} {     lassign [split $line ","] date time;   # ignore other fields...     set ts [tidyuptimestamp $date $time]     if {[info exist map($ts)]} {         # simple concatenation!         puts $fout "$line,$map($ts)"     } } close $fout close $fin 

if dates , times not precisely equal, you'll need cleaning above code works entirely on textual representation of everything. pop tidyuptimestamp procedure…