javascript - How best to handle relative urls in scraped content? -


what methods make relative urls absolute in scraped content scraped html appears original , css not broken?

i found out <base> tag may help. how can find out original base of url is?

i don't care interactions links, want them appear correct.

assume site 'example.com/blog/new/i.html' scrape has 2 resources

  1. < link src="/style/style.css" >
  2. < link src="newstyle.css" >.

now if set base 'example.com/blog/new/i.html' wont first 1 break

keep track of url of each page scraped. 1 way save full url filename. then, can resolve relative urls per html spec.


Comments

Post a Comment

Popular posts from this blog

c++ - CryptStringToBinary API behavior -

i seeing strange behavior cryptstringtobinary cryptography api. please see below code (config: x64 debug): #include "stdafx.h" #include <windows.h> #include <strsafe.h> #include <iostream> #include <exception> void main() { dword dwskip; dword dwflags; dword dwdatalen; //lpcwstr pszinput = l"jaaaaaecaaadzgaaakqaagdnnl1l56bwgfjdgr3rpxtqqqn6daw3usv2emkjym4t"; //this works fine lpcwstr pszinput = l"mytest"; //doesnt work, api returns false,error code 0x0000000d // determine size of byte array , allocate memory. if(! cryptstringtobinary( pszinput, _tcslen( pszinput ) + 1, crypt_string_base64, null, &dwdatalen, &dwskip, &dwflags ) ) { dword dw = getlasterror(); //0x0000000d: data invalid throw std::exception( "error computing byte length." ); } byte *pbytebyte = null; try { ...
Read more

c++ - Correct method for redrawing a layered window -

i have window created ws_ex_layered window style. drawing onto memory bitmap using gdi+, , using updatelayeredwindow update graphical content of layered window. intend use window main window of application, require redraw frequently. seeing layered windows not receive wm_paint windows message [?] , need come appropriate method re-drawing window. optimisation not essential , it's nice have cake , eat too. therefore, in search of "correct" method use. here thoughts far: i guess it's idea render onto off-screen bitmap before bitblt ing or similar. 60 frames rendered per second should (more than?) enough (but how compare other applications' frame rates?). possible solutions: use settimer send wm_timer message on regular basis. useful because through specifying time-out value, can achieve desired frames per second, without requirement measure duration "frame" takes rendered. would cause input or other lags due frequency , speed of m...
Read more

java.util.scanner - How to read and add only numbers to array from a text file -

i'm learning java , have question regarding reading file want read numbers file contains strings well. here example of file: 66.56 "3 java 3-43 5-42 2.1 1 and here coding: public class test { public static void main (string [] args){ if (0 < args.length) { file x = new file(args[0]); try{ scanner in = new scanner( new fileinputstream(x)); arraylist<double> test = new arraylist<>(); while(in.hasnext()){ if(in.hasnextdouble()){ double f=in.nextdouble(); test.add(f);} else {in.next();} } catch(ioexception e) { system.err.println("exception during reading: " + e); } } my problem add 66.56,2.1 , 1 doesn't add 3 after "3 or ignores 3-43 , 5-42 can tell me how skip strings , add doubles here? thanks all said 3 ie; "3, 3-43 , 4-42 strings either u read string , split , check number @ " , - or put in space between characters , integers. jvm after compilation tre...
Read more