41 Commits

Author	SHA1	Message	Date
HugoPoi	c58d4fa74d	fix(proxy): throw on use_proxies_only if no proxies given	2020-01-17 15:55:17 +01:00
HugoPoi	89dc5c3ebb	fix(scrape-manager): conflict between proxies and user_agent option	2020-01-17 12:07:12 +01:00
HugoPoi	28332528ea	test(duckduckgo): implement tests for duckduckgo module	2020-01-15 16:33:30 +01:00
HugoPoi	b685fb4def	test: working test for html_output	2020-01-10 09:51:54 +01:00
HugoPoi	394b567db6	test: add user_agent tests, add html_output tests	2020-01-10 09:35:24 +01:00
HugoPoi	cac6b87e92	test: Bing tests working, refactor proxy for tests	2020-01-08 14:40:28 +01:00
HugoPoi	1c1db88545	test: add config proxy options tests	2020-01-07 16:50:09 +01:00
HugoPoi	f192e4ebb4	test: remove legacy tests	2020-01-07 16:43:17 +01:00
HugoPoi	3ab8e46126	test: add bing module test	2020-01-07 09:48:46 +01:00
HugoPoi	392c43390e	test(google): add real integration/unit tests for google module	2020-01-03 19:21:34 +01:00
Nikolai Tschacher	1694ee92d0	updated to puppeteeer 2.0	2019-11-08 16:21:16 +01:00
Nikolai Tschacher	5e47c27c70	too late to find a proper commit description	2019-09-23 23:38:38 +02:00
Nikolai Tschacher	95a5ee56d8	remove cheerio from parsing	2019-09-23 21:57:13 +02:00
Nikolai Tschacher	4b581bd03f	removed static tests because they are too larege	2019-09-13 18:21:17 +02:00
Nikolai Tschacher	21378dab02	removed some search engines, added tests for existing, added yandex search engines	2019-09-13 16:15:33 +02:00
Nikolai Tschacher	e661241f6f	added some parsing to google	2019-08-16 20:10:40 +02:00
Nikolai Tschacher	19a172c654	better tests	2019-08-13 15:28:30 +02:00
Nikolai Tschacher	ca941cee45	added static bing test, added html cleaning when exporting html	2019-08-12 16:05:17 +02:00
Nikolai Tschacher	0427d9f915	Merge branch 'master' into error-management	2019-08-12 00:04:27 +02:00
Nikolai Tschacher	87fcdd35d5	readme in static tests	2019-08-12 00:01:02 +02:00
Nikolai Tschacher	4ca50ab2b9	added new static test case that runs much faster and tests a lot of behavior	2019-08-11 23:58:10 +02:00
HugoPoi	50bda275a6	Clean integration tests for mocha	2019-08-05 17:01:48 +02:00
Nikolai Tschacher	59154694f2	fixed issue https://github.com/NikolaiT/se-scraper/issues/37	2019-07-18 19:14:33 +02:00
Nikolai Tschacher	a413cb54ef	parsing ads works for duckduckgo, google, bing. tested.	2019-07-07 19:38:28 +02:00
Nikolai Tschacher	bbebe3ce60	parsing ads is supported now for google, bing and duckduckgo	2019-07-06 21:42:13 +02:00
HugoPoi	d9ac9f4162	Add test for html_output, refactor the results return	2019-06-26 12:03:42 +02:00
Nikolai Tschacher	db5fbb23d2	removed unnecessary sleeping times	2019-06-12 18:14:49 +02:00
Nikolai Tschacher	5bf7c94b9a	new version	2019-06-11 22:01:27 +02:00
Nikolai Tschacher	6825c97790	changed api big time	2019-06-11 18:16:59 +02:00
Nikolai Tschacher	7a8c6f13f0	fixed #11 by improving baidu a lot in speed and quality	2019-03-14 23:33:46 +01:00
Nikolai Tschacher	51d617442d	added support for amazon	2019-03-10 20:02:42 +01:00
Nikolai Tschacher	62b3b688b4	minor fixes	2019-03-07 13:16:12 +01:00
Nikolai Tschacher	79d32a315a	fixed some errors and way better README	2019-02-28 15:34:25 +01:00
Nikolai Tschacher	7572ebd314	added chrome detection evasion techniques	2019-02-07 16:09:38 +01:00
Nikolai Tschacher	d5b147296e	ticker search OOP now and added tests	2019-01-31 22:13:22 +01:00
Nikolai Tschacher	d35a602994	added clean test cases for bing and duckduckgo	2019-01-31 15:36:27 +01:00
Nikolai Tschacher	7441c57a43	removed generic tests. too complicated	2019-01-31 14:58:07 +01:00
Nikolai Tschacher	c60d0f3528	clean test case for google is passing	2019-01-31 14:57:34 +01:00
Nikolai Tschacher	987e3d7342	tested and works	2019-01-30 23:53:09 +01:00
Nikolai Tschacher	8e695626b6	added pluggable functionality	2019-01-27 15:54:56 +01:00
Nikolai Tschacher	9af1630e19	initial	2018-12-24 14:25:02 +01:00