Python: Routine for handling API request limits -


i need make lot of http requests (> 1000) against public service allows 500 http requests per day. hence, have count number of executed requests , stop when reach maximum daily amount continue next day remaining calls. in particular, iterate on non-sorted list, cannot assume elements in order. code looks this:

from requests import session, request  request_parameters = {'api_key': api_key}  user_id in all_user_ids:     r = requests.get('http://public-api.com/%s'% user_id, request_parameters)     text = r.content     # stuff text

is there package or pattern can recommend counting , resuming api calls this?

i suggest implementing simple counter stop when have hit limit day along local cache of data have received. when run process again next day check each record against local cache first , go on call web service if there no record in local cache. way have of data unless generating more requests per day service usage limit.

the format of cache depend on returned web service , how of data need, may simple csv file unique identifier search against , other fields need retrieve in future.

another option store whole response each call (if need lot of it) in dictionary key being unique identifier , value being response. saved json file , loaded memory checking against on future runs.


Comments

Post a Comment

Popular posts from this blog

java.util.scanner - How to read and add only numbers to array from a text file -

i'm learning java , have question regarding reading file want read numbers file contains strings well. here example of file: 66.56 "3 java 3-43 5-42 2.1 1 and here coding: public class test { public static void main (string [] args){ if (0 < args.length) { file x = new file(args[0]); try{ scanner in = new scanner( new fileinputstream(x)); arraylist<double> test = new arraylist<>(); while(in.hasnext()){ if(in.hasnextdouble()){ double f=in.nextdouble(); test.add(f);} else {in.next();} } catch(ioexception e) { system.err.println("exception during reading: " + e); } } my problem add 66.56,2.1 , 1 doesn't add 3 after "3 or ignores 3-43 , 5-42 can tell me how skip strings , add doubles here? thanks all said 3 ie; "3, 3-43 , 4-42 strings either u read string , split , check number @ " , - or put in space between characters , integers. jvm after compilation tre...
Read more

rewrite - Trouble with Wordpress multiple custom querystrings -

i'm working on adding multiple querystring , doesn't seem work. i've found codes seems work others might i'm missing something. i want like: http://mydomain.com/board/?getyear=2013&getsometext2=sometext to http://mydomain.com/board/2013/sometext/ i added code below functions.php file. doesn't seem work should though. this works. echoing querystring gets both variable without problem. http://mydomain.com/board/2013/sometext/ but doesn't. tested querystring vis isset() seems nothing being set/get http://mydomain.com/board/2013/ function add_query_vars($vars) { $vars[] = "getyear"; $vars[] = "getsometext2"; return $vars; } add_filter('query_vars', 'add_query_vars'); function add_rewrite_rules($arules) { $anewrules = array('board/([^/]+)/([^/]+)/?$' => 'index.php?pagename=board&getyear=$matches[1]&getsometext2=$matches[2]'); $arules = $a...
Read more