42 Commits

Author	SHA1	Message	Date
HugoPoi	bcd181111b	refactor(log): remove common.js, use winston and debug	2019-12-15 17:56:22 +01:00
Nikolai Tschacher	4a3a0e6fd4	better pluggable api	2019-10-05 19:39:33 +02:00
Nikolai Tschacher	5e47c27c70	too late to find a proper commit description	2019-09-23 23:38:38 +02:00
Nikolai Tschacher	4b581bd03f	removed static tests because they are too larege	2019-09-13 18:21:17 +02:00
Nikolai Tschacher	21378dab02	removed some search engines, added tests for existing, added yandex search engines	2019-09-13 16:15:33 +02:00
Nikolai Tschacher	19a172c654	better tests	2019-08-13 15:28:30 +02:00
Nikolai Tschacher	ca941cee45	added static bing test, added html cleaning when exporting html	2019-08-12 16:05:17 +02:00
Nikolai Tschacher	78fe12390b	better user agents now, added option to include screenshots as base64 in results	2019-07-18 20:19:15 +02:00
Nikolai Tschacher	fcbe66b56b	using random user agents now from https://github.com/intoli/user-agents	2019-07-18 19:34:09 +02:00
Nikolai Tschacher	59154694f2	fixed issue https://github.com/NikolaiT/se-scraper/issues/37	2019-07-18 19:14:33 +02:00
Nikolai Tschacher	60a9d52924	add fucking google product information	2019-07-11 19:23:40 +02:00
Nikolai Tschacher	dab25f9068	added google shopping results	2019-07-11 16:42:01 +02:00
Nikolai Tschacher	a413cb54ef	parsing ads works for duckduckgo, google, bing. tested.	2019-07-07 19:38:28 +02:00
Nikolai Tschacher	bbebe3ce60	parsing ads is supported now for google, bing and duckduckgo	2019-07-06 21:42:13 +02:00
Nikolai Tschacher	d1e9b21269	added google maps scraper	2019-06-29 17:00:19 +02:00
Nikolai Tschacher	0d7f6dcd11	worked on issue #31	2019-06-18 22:23:52 +02:00
Nikolai Tschacher	80d23a9d57	users may pass their own user agents, different browsers have random user agents and not the same now	2019-06-17 21:25:45 +02:00
Nikolai Tschacher	0c9f353cb2	remove hardcoded sleep() in Google Image	2019-06-17 00:03:13 +02:00
Nikolai Tschacher	43d5732de7	resolved issue #30, custom scrapers now possible. new npm version	2019-06-13 12:34:39 +02:00
Nikolai Tschacher	784e887787	fixed issue #22	2019-06-12 21:25:20 +02:00
Nikolai Tschacher	5bf7c94b9a	new version	2019-06-11 22:01:27 +02:00
Nikolai Tschacher	35943e7449	minor stuff	2019-06-11 18:33:11 +02:00
Nikolai Tschacher	7e06944fa1	updated README	2019-06-11 18:27:34 +02:00
Nikolai Tschacher	6825c97790	changed api big time	2019-06-11 18:16:59 +02:00
Nikolai Tschacher	3d69f4e249	added a proxy test script	2019-05-06 21:54:23 +02:00
Nikolai Tschacher	1593759556	passing chrome flags directly now possible	2019-04-01 15:33:26 +02:00
Nikolai Tschacher	775dcfa077	proxy mgmt better	2019-03-22 18:55:17 +01:00
Nikolai Tschacher	b82c769bb1	google_news_old supports google_news_old_settings now	2019-03-20 15:28:04 +01:00
Nikolai Tschacher	7a8c6f13f0	fixed #11 by improving baidu a lot in speed and quality	2019-03-14 23:33:46 +01:00
Nikolai Tschacher	51d617442d	added support for amazon	2019-03-10 20:02:42 +01:00
Nikolai Tschacher	dd1f36076e	can now parse args from string to json	2019-03-07 15:50:36 +01:00
Nikolai Tschacher	7b52b4e62f	added suport for custom query string parameters	2019-03-06 00:08:25 +01:00
Nikolai Tschacher	8cbf37eaba	minor improvements	2019-03-02 22:32:26 +01:00
Nikolai Tschacher	79d32a315a	fixed some errors and way better README	2019-02-28 15:34:25 +01:00
Nikolai Tschacher	089e410ec6	support for multible browsers and proxies	2019-02-27 20:58:13 +01:00
Nikolai Tschacher	7572ebd314	added chrome detection evasion techniques	2019-02-07 16:09:38 +01:00
Nikolai Tschacher	987e3d7342	tested and works	2019-01-30 23:53:09 +01:00
Nikolai Tschacher	89441070cd	before_keyword_scraped() hook supported	2019-01-29 13:29:24 +01:00
Nikolai Tschacher	a20b6c79a9	added handle_results	2019-01-27 20:08:09 +01:00
Nikolai Tschacher	86a66a09fd	pluggable is now class	2019-01-27 19:08:07 +01:00
Nikolai Tschacher	8e695626b6	added pluggable functionality	2019-01-27 15:54:56 +01:00
Nikolai Tschacher	b354e6918d	.	2019-01-26 20:15:19 +01:00