2017-04-10 00:56:52 [scrapy.utils.log] INFO: Scrapy 1.3.3started (bot: firstCrawler)2017-04-10 00:56:52 [scrapy.utils.log] INFO: Overridden settings: {‘NEWSPIDER_MODULE’: ‘firstCrawler.spiders’, ‘SPIDER_MODULES’: [‘firstCrawler.spiders’], ‘ROBOTSTXT_OBEY’: True, ‘BOT_NAME’: ‘firstCrawler’}2017-04-10 00:56:52[scrapy.middleware] INFO: Enabled extensions:
[‘scrapy.extensions.logstats.LogStats’,’scrapy.extensions.telnet.TelnetConsole’,’scrapy.extensions.corestats.CoreStats’]2017-04-10 00:56:52[scrapy.middleware] INFO: Enabled downloader middlewares:
[‘scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware’,’scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware’,’scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware’,’scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware’,’scrapy.downloadermiddlewares.useragent.UserAgentMiddleware’,’scrapy.downloadermiddlewares.retry.RetryMiddleware’,’scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware’,’scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware’,’scrapy.downloadermiddlewares.redirect.RedirectMiddleware’,’scrapy.downloadermiddlewares.cookies.CookiesMiddleware’,’scrapy.downloadermiddlewares.stats.DownloaderStats’]2017-04-10 00:56:52[scrapy.middleware] INFO: Enabled spider middlewares:
[‘scrapy.spidermiddlewares.httperror.HttpErrorMiddleware’,’scrapy.spidermiddlewares.offsite.OffsiteMiddleware’,’scrapy.spidermiddlewares.referer.RefererMiddleware’,’scrapy.spidermiddlewares.urllength.UrlLengthMiddleware’,’scrapy.spidermiddlewares.depth.DepthMiddleware’]2017-04-10 00:56:52[scrapy.middleware] INFO: Enabled item pipelines:
[]2017-04-10 00:56:52[scrapy.core.engine] INFO: Spider opened2017-04-10 00:56:52 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)2017-04-10 00:56:52 [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023
2017-04-10 00:56:55 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None)
2017-04-10 00:56:56 [scrapy.core.engine] DEBUG: Crawled (200) (referer: None)
[u’IT168\u4e91\u8ba1\u7b97\u9891\u9053′] [u’http://cloud.it168.com/’] [u’\r\n\t\t\t\r\n \u63d0\u4f9b\u4e91\u8ba1\u7b97\u6700\u65b0\u8d44\u8baf\u3001\u6280\u672f\u89e3\u8bfb\uff0c\u4ea7\u54c1\u548c\u670d\u52a1\u7684\u5bfc\u8d2d\u548c\u9009\u578b\uff0c\u5206\u4eab\u56fd\u5185\u5916\u4e91\u8ba1\u7b97\u7684\u6210\u529f\u6848\u4f8b\u3002\r\n’, u’\r\n’]
[u’\u4e2d\u4e91\u7f51′] [u’http://www.china-cloud.com/’] [u’\r\n\t\t\t\r\n \u63d0\u4f9b\u4e91\u8ba1\u7b97\u8d44\u8baf\uff0c\u5305\u62ec\u4e91\u8ba1\u7b97\u6280\u672f\u3001\u4e91\u8ba1\u7b97\u5e94\u7528\u3001\u4e91\u8ba1\u7b97\u4ea7\u4e1a\u3001\u4e91\u8ba1\u7b97\u5e02\u573a\u3001\u4e91\u5b58\u50a8\u3001\u4e91\u5b89\u5168\u3001\u4e91\u670d\u52a1\u7b49\u76f8\u5173\u8d44\u8baf\u3002\r\n’, u’\r\n’]
[u’\u4e2d\u5173\u6751\u5728\u7ebf\u4e91\u8ba1\u7b97\u9891\u9053′] [u’http://cloud.zol.com.cn/’] [u’\r\n\t\t\t\r\n \u63d0\u4f9b\u4e91\u8ba1\u7b97\u884c\u60c5\uff0c\u4e91\u8ba1\u7b97\u62a5\u4ef7\uff0c\u4e91\u8ba1\u7b97\u8bc4\u6d4b\uff0c\u4e91\u8ba1\u7b97\u8bba\u575b\uff0c\u4e91\u8ba1\u7b97\u7ecf\u9500\u5546\u7b49\u6709\u4ef7\u503c\u7684\u53c2\u8003\u3002\r\n’, u’\r\n’]
[u’\u4e2d\u56fd\u4e91\u8ba1\u7b97′] [u’http://www.chinacloud.cn/’] [u’\r\n\t\t\t\r\n \u63d0\u4f9b\u4e91\u8ba1\u7b97\u6982\u5ff5\u3001\u4e91\u8ba1\u7b97\u6280\u672f\u3001\u4e91\u8ba1\u7b97\u5e94\u7528\u3001\u4e91\u8ba1\u7b97\u5b66\u4e60\u3001\u4e91\u8ba1\u7b97\u673a\u3001\u4e91\u5b58\u50a8\u3001\u4e91\u5b89\u5168\u3001\u4e91\u670d\u52a1\u7b49\u4ea4\u6d41\u4e2d\u5fc3\u3002\r\n’, u’\r\n’]
[u’\u4e2d\u56fd\u4e91\u8ba1\u7b97\u670d\u52a1\u7f51′] [u’http://www.cloudguide.com.cn/’] [u’\r\n\t\t\t\r\n \u63d0\u4f9b\u4e91\u8ba1\u7b97\u65b0\u95fb\u8d44\u8baf\u4ee5\u53ca\u4e91\u8ba1\u7b97\u4ea7\u54c1\u548c\u670d\u52a1\uff0c\u540c\u65f6\u63d0\u4f9b\u6709\u4ef7\u503c\u7684\u4e91\u8ba1\u7b97\u884c\u4e1a\u89e3\u51b3\u65b9\u6848\u53ca\u5e94\u7528\u6848\u4f8b\u7b49\u70ed\u95e8\u8d44\u6e90 \u3002\r\n’, u’\r\n’]2017-04-10 00:56:56[scrapy.core.engine] INFO: Closing spider (finished)2017-04-10 00:56:56[scrapy.statscollectors] INFO: Dumping Scrapy stats:
{‘downloader/request_bytes’: 549,’downloader/request_count’: 2,’downloader/request_method_count/GET’: 2,’downloader/response_bytes’: 6789,’downloader/response_count’: 2,’downloader/response_status_count/200′: 2,’finish_reason’: ‘finished’,’finish_time’: datetime.datetime(2017, 4, 9, 16, 56, 56, 295317),’log_count/DEBUG’: 3,’log_count/INFO’: 7,’response_received_count’: 2,’scheduler/dequeued’: 1,’scheduler/dequeued/memory’: 1,’scheduler/enqueued’: 1,’scheduler/enqueued/memory’: 1,’start_time’: datetime.datetime(2017, 4, 9, 16, 56, 52, 724189)}2017-04-10 00:56:56 [scrapy.core.engine] INFO: Spider closed (finished)
Original: https://blog.csdn.net/weixin_30022703/article/details/113639236
Author: 能量中国
Title: python scarpy文献_python爬虫scrapy学习之篇二
原创文章受到原创版权保护。转载请注明出处：https://www.johngo689.com/792674/
转载文章受原作者版权保护。转载请注明原作者出处！