extern/se-scraper
1
0
Fork 1
mirror of https://github.com/NikolaiT/se-scraper.git synced 2024-11-22 07:33:07 +01:00
Code Issues Packages Projects Releases Wiki Activity
88 Commits 14 Branches 0 Tags 12 MiB
e661241f6f
Commit Graph

28 Commits

Author SHA1 Message Date
Nikolai Tschacher
98414259fe docker support added 2019-08-13 17:35:06 +02:00
Nikolai Tschacher
4ca50ab2b9 added new static test case that runs much faster and tests a lot of behavior 2019-08-11 23:58:10 +02:00
Nikolai Tschacher
78fe12390b better user agents now, added option to include screenshots as base64 in results 2019-07-18 20:19:15 +02:00
Nikolai Tschacher
d1e9b21269 added google maps scraper 2019-06-29 17:00:19 +02:00
Nikolai Tschacher
0d7f6dcd11 worked on issue #31 2019-06-18 22:23:52 +02:00
Nikolai Tschacher
43d5732de7 resolved issue #30, custom scrapers now possible. new npm version 2019-06-13 12:34:39 +02:00
Nikolai Tschacher
5bf7c94b9a new version 2019-06-11 22:01:27 +02:00
Nikolai Tschacher
d4d06f7d67 need to edit readme 2019-06-11 18:34:51 +02:00
Nikolai Tschacher
35943e7449 minor stuff 2019-06-11 18:33:11 +02:00
Nikolai Tschacher
7e06944fa1 updated README 2019-06-11 18:27:34 +02:00
Nikolai Tschacher
1593759556 passing chrome flags directly now possible 2019-04-01 15:33:26 +02:00
Nikolai Tschacher
7a8c6f13f0 fixed #11 by improving baidu a lot in speed and quality 2019-03-14 23:33:46 +01:00
Nikolai Tschacher
51d617442d added support for amazon 2019-03-10 20:02:42 +01:00
Nikolai Tschacher
7b52b4e62f added suport for custom query string parameters 2019-03-06 00:08:25 +01:00
Nikolai Tschacher
7239e23cba fixed pluggable 2019-03-03 16:46:10 +01:00
Nikolai Tschacher
abf4458e46 fixed quotes in user agent. this lead to cloudflare detecting the scraper. very bad. 2019-03-01 16:02:30 +01:00
Nikolai Tschacher
79d32a315a fixed some errors and way better README 2019-02-28 15:34:25 +01:00
Nikolai Tschacher
089e410ec6 support for multible browsers and proxies 2019-02-27 20:58:13 +01:00
Nikolai Tschacher
77c332d7c8 updated readme 2019-02-07 16:26:11 +01:00
Nikolai Tschacher
7572ebd314 added chrome detection evasion techniques 2019-02-07 16:09:38 +01:00
Nikolai Tschacher
d5b147296e ticker search OOP now and added tests 2019-01-31 22:13:22 +01:00
Nikolai Tschacher
987e3d7342 tested and works 2019-01-30 23:53:09 +01:00
Nikolai Tschacher
9e62f23451 resolved some issues. proxy possible now. scraping for more than one page possible now 2019-01-29 22:48:08 +01:00
Nikolai Tschacher
8e695626b6 added pluggable functionality 2019-01-27 15:54:56 +01:00
Nikolai Tschacher
e78d7145b5 faster scraping, added ticker search engines 2019-01-27 01:27:52 +01:00
Nikolai Tschacher
b354e6918d . 2019-01-26 20:15:19 +01:00
Nikolai Tschacher
bab902e80a supporting yahoo ticker search for news 2019-01-24 15:50:03 +01:00
Nikolai Tschacher
9af1630e19 initial 2018-12-24 14:25:02 +01:00