performance - Best approach to storing scientific data sets on disk C++ -


i'm working on project requires working gigabytes of scientific data sets. data sets in form of large arrays (30,000 elements) of integers , floating point numbers. problem here large fit memory, need on disk solution storing , working them. make problem more fun, restricted using 32-bit architecture (as work) , need try maximize performance solution.

so far, i've worked hdf5, worked okay, found little complicated work with. so, thought next best thing try nosql database, couldn't find way store arrays in database short of casting them character arrays , storing them that, caused lot of bad pointer headaches.

so, i'd know guys recommend. maybe have less painful way of working hdf5 while @ same time maximizing performance. or maybe know of nosql database works storing type of data. or maybe i'm going in totally wrong direction , you'd smack sense me.

anyway, i'd appreciate words of wisdom guys can offer me :)

assuming data sets large enough merit (e.g., instead of 30,000 elements, 30,000x30,000 array of doubles), might want consider stxxl. provides interfaces intended (and largely succeed at) imitate of collections in c++ standard library, intended work data large fit in memory.


Comments

Post a Comment

Popular posts from this blog

java.util.scanner - How to read and add only numbers to array from a text file -

i'm learning java , have question regarding reading file want read numbers file contains strings well. here example of file: 66.56 "3 java 3-43 5-42 2.1 1 and here coding: public class test { public static void main (string [] args){ if (0 < args.length) { file x = new file(args[0]); try{ scanner in = new scanner( new fileinputstream(x)); arraylist<double> test = new arraylist<>(); while(in.hasnext()){ if(in.hasnextdouble()){ double f=in.nextdouble(); test.add(f);} else {in.next();} } catch(ioexception e) { system.err.println("exception during reading: " + e); } } my problem add 66.56,2.1 , 1 doesn't add 3 after "3 or ignores 3-43 , 5-42 can tell me how skip strings , add doubles here? thanks all said 3 ie; "3, 3-43 , 4-42 strings either u read string , split , check number @ " , - or put in space between characters , integers. jvm after compilation tre...
Read more

rewrite - Trouble with Wordpress multiple custom querystrings -

i'm working on adding multiple querystring , doesn't seem work. i've found codes seems work others might i'm missing something. i want like: http://mydomain.com/board/?getyear=2013&getsometext2=sometext to http://mydomain.com/board/2013/sometext/ i added code below functions.php file. doesn't seem work should though. this works. echoing querystring gets both variable without problem. http://mydomain.com/board/2013/sometext/ but doesn't. tested querystring vis isset() seems nothing being set/get http://mydomain.com/board/2013/ function add_query_vars($vars) { $vars[] = "getyear"; $vars[] = "getsometext2"; return $vars; } add_filter('query_vars', 'add_query_vars'); function add_rewrite_rules($arules) { $anewrules = array('board/([^/]+)/([^/]+)/?$' => 'index.php?pagename=board&getyear=$matches[1]&getsometext2=$matches[2]'); $arules = $a...
Read more