Commit Graph

24 Commits

Author SHA1 Message Date
Nikolai Tschacher
3d69f4e249 added a proxy test script 2019-05-06 21:54:23 +02:00
Nikolai Tschacher
1593759556 passing chrome flags directly now possible 2019-04-01 15:33:26 +02:00
Nikolai Tschacher
b82c769bb1 google_news_old supports google_news_old_settings now 2019-03-20 15:28:04 +01:00
Nikolai Tschacher
7a8c6f13f0 fixed #11 by improving baidu a lot in speed and quality 2019-03-14 23:33:46 +01:00
Nikolai Tschacher
51d617442d added support for amazon 2019-03-10 20:02:42 +01:00
Nikolai Tschacher
dd1f36076e can now parse args from string to json 2019-03-07 15:50:36 +01:00
Nikolai Tschacher
7b52b4e62f added suport for custom query string parameters 2019-03-06 00:08:25 +01:00
Nikolai Tschacher
7239e23cba fixed pluggable 2019-03-03 16:46:10 +01:00
Nikolai Tschacher
8cbf37eaba minor improvements 2019-03-02 22:32:26 +01:00
Nikolai Tschacher
abf4458e46 fixed quotes in user agent. this lead to cloudflare detecting the scraper. very bad. 2019-03-01 16:02:30 +01:00
Nikolai Tschacher
79d32a315a fixed some errors and way better README 2019-02-28 15:34:25 +01:00
Nikolai Tschacher
089e410ec6 support for multible browsers and proxies 2019-02-27 20:58:13 +01:00
Nikolai Tschacher
7b5048b8ee num_keywords are counted now. added to pluggable 2019-02-07 16:21:56 +01:00
Nikolai Tschacher
7572ebd314 added chrome detection evasion techniques 2019-02-07 16:09:38 +01:00
Nikolai Tschacher
c60d0f3528 clean test case for google is passing 2019-01-31 14:57:34 +01:00
Nikolai Tschacher
9e62f23451 resolved some issues. proxy possible now. scraping for more than one page possible now 2019-01-29 22:48:08 +01:00
Nikolai Tschacher
89441070cd before_keyword_scraped() hook supported 2019-01-29 13:29:24 +01:00
Nikolai Tschacher
c5e3e84e1d minor changes 2019-01-27 22:11:41 +01:00
Nikolai Tschacher
a20b6c79a9 added handle_results 2019-01-27 20:08:09 +01:00
Nikolai Tschacher
8e695626b6 added pluggable functionality 2019-01-27 15:54:56 +01:00
Nikolai Tschacher
e78d7145b5 faster scraping, added ticker search engines 2019-01-27 01:27:52 +01:00
Nikolai Tschacher
b354e6918d . 2019-01-26 20:15:19 +01:00
Nikolai Tschacher
bab902e80a supporting yahoo ticker search for news 2019-01-24 15:50:03 +01:00
Nikolai Tschacher
9af1630e19 initial 2018-12-24 14:25:02 +01:00