c# - Enumerating Datatable rows in batches efficiently -


i'd enumerate through datatable in batches. so, i've created method returns ienumerable<datatable> method looks this:

public ienumerable<datatable> enumeraterowsinbatches( datatable table, int batchsize ) {      int rowcount = table.rows.count;     int batchindex = 0;     while( batchindex * batchsize < rowcount ) {         datatable result = table.clone();         int batchstart = batchindex * batchsize;         int batchlimit = ( batchindex + 1 ) * batchsize;         if( rowcount < batchlimit )             batchlimit = rowcount;         for( int = batchstart; < batchlimit; i++ ) {             result.importrow( table.rows[ ] );         }         batchindex++;         yield return result;     } }

this works pretty nicely actually. i'm iterating through these batches in order send sql server using table valued parameter. i'm seeing importrow taking majority of elapsed time , i'd speed up.

i'm looking how that. i'm free treat data read-only , sense copying rows not strictly necessary here.

i came approach lead ~40% performance improvement in tests:

public static ienumerable<datatable> enumeraterowsinbatches(datatable table,                                                             int batchsize) {     int rowcount = table.rows.count;     int batchindex = 0;     datatable result = table.clone(); // not change, avoid recreate     while (batchindex * batchsize < rowcount)     {         result.rows.clear(); // reuse datatable, clear previous results         int batchstart = batchindex * batchsize;         int batchlimit = (batchindex + 1) * batchsize;         if (rowcount < batchlimit)             batchlimit = rowcount;          (int = batchstart; < batchlimit; i++)             result.rows.add(table.rows[i].itemarray); // avoid importrow          batchindex++;         yield return result;     } }

Comments

Post a Comment

Popular posts from this blog

c++ - CryptStringToBinary API behavior -

i seeing strange behavior cryptstringtobinary cryptography api. please see below code (config: x64 debug): #include "stdafx.h" #include <windows.h> #include <strsafe.h> #include <iostream> #include <exception> void main() { dword dwskip; dword dwflags; dword dwdatalen; //lpcwstr pszinput = l"jaaaaaecaaadzgaaakqaagdnnl1l56bwgfjdgr3rpxtqqqn6daw3usv2emkjym4t"; //this works fine lpcwstr pszinput = l"mytest"; //doesnt work, api returns false,error code 0x0000000d // determine size of byte array , allocate memory. if(! cryptstringtobinary( pszinput, _tcslen( pszinput ) + 1, crypt_string_base64, null, &dwdatalen, &dwskip, &dwflags ) ) { dword dw = getlasterror(); //0x0000000d: data invalid throw std::exception( "error computing byte length." ); } byte *pbytebyte = null; try { ...
Read more

c++ - Correct method for redrawing a layered window -

i have window created ws_ex_layered window style. drawing onto memory bitmap using gdi+, , using updatelayeredwindow update graphical content of layered window. intend use window main window of application, require redraw frequently. seeing layered windows not receive wm_paint windows message [?] , need come appropriate method re-drawing window. optimisation not essential , it's nice have cake , eat too. therefore, in search of "correct" method use. here thoughts far: i guess it's idea render onto off-screen bitmap before bitblt ing or similar. 60 frames rendered per second should (more than?) enough (but how compare other applications' frame rates?). possible solutions: use settimer send wm_timer message on regular basis. useful because through specifying time-out value, can achieve desired frames per second, without requirement measure duration "frame" takes rendered. would cause input or other lags due frequency , speed of m...
Read more

java.util.scanner - How to read and add only numbers to array from a text file -

i'm learning java , have question regarding reading file want read numbers file contains strings well. here example of file: 66.56 "3 java 3-43 5-42 2.1 1 and here coding: public class test { public static void main (string [] args){ if (0 < args.length) { file x = new file(args[0]); try{ scanner in = new scanner( new fileinputstream(x)); arraylist<double> test = new arraylist<>(); while(in.hasnext()){ if(in.hasnextdouble()){ double f=in.nextdouble(); test.add(f);} else {in.next();} } catch(ioexception e) { system.err.println("exception during reading: " + e); } } my problem add 66.56,2.1 , 1 doesn't add 3 after "3 or ignores 3-43 , 5-42 can tell me how skip strings , add doubles here? thanks all said 3 ie; "3, 3-43 , 4-42 strings either u read string , split , check number @ " , - or put in space between characters , integers. jvm after compilation tre...
Read more