Scrapyd(centos 部署scrapy)安装介绍以及sqllite3配置

Scrapyd(centos 部署scrapy)安装介绍以及sqllite3配置

3)启动scrapyd时,提示错误:No module named ‘_sqlite3’ 没有找到sqlite3,安装sqlite3命令如下:

cd tmp
wget https://www.sqlite.org/2022/sqlite-autoconf-3390100.tar.gz
tar -xvzf sqlite-autoconf-3390100.tar.gz
cd sqlite-autoconf-3390100
#配置,注意这里指定了路径后,下面有地方配置需要对应:
./configure --prefix=/usr/local/bin/sqlite
#编译和安装
make -j4&&sudo make install

检查sqlite3是否完装完成,输入sqlite3 显示当前的版本,按.exit 命令退出

Scrapyd(centos 部署scrapy)安装介绍以及sqllite3配置

sqlite3安装版本,参考: https://sqlite.org/index.html

所有下载版本地址:https://sqlite.org/chronology.html

4)安装sqlite3后,启动再scrapyd还是不行,查找_sqlite3.so文件发现,在python2.7里面

[root@iZwz927xptl8kw8zqrmwvcZ lib-dynload]# find / -name _sqlite3.so
/usr/local/aegis/PythonLoader/lib/python2.7/lib-dynload/_sqlite3.so

5)修改/tmp/Python-3.10.5/setup.py文件,找到 sqlite_inc_paths,在其中添加sqlite3安装路径,数组后面二项是新增的

sqlite_inc_paths = [ '/usr/include',
                             '/usr/include/sqlite',
                             '/usr/include/sqlite3',
                             '/usr/local/include',
                             '/usr/local/include/sqlite',
                             '/usr/local/include/sqlite3',
                             '/usr/local/bin/sqlite/include',
                             '/usr/local/bin/sqlite/include/sqlite3'
                             ]

6)再次编译安装python3, cd到tmp/python-3.10.5下进行配置,接着编译安装

[root@iZwz927xptl8kw8zqrmwvcZ Python-3.10.5]# ./configure --prefix=/usr/local/bin/python310 --enable-loadable-sqlite-extensions
[root@iZwz927xptl8kw8zqrmwvcZ Python-3.10.5]# make && sudo make install

再次查找_sqlite3.so,发现多了文件:

[root@iZwz927xptl8kw8zqrmwvcZ Python-3.10.5]#  find / -name _sqlite*.so
/usr/local/bin/python310/lib/python3.10/lib-dynload/_sqlite3.cpython-310-x86_64-linux-gnu.so
/usr/local/aegis/PythonLoader/lib/python2.7/lib-dynload/_sqlite3.so
/tmp/Python-3.10.5/build/lib.linux-x86_64-3.10/_sqlite3.cpython-310-x86_64-linux-gnu.so

另一种测试python3使用sqlite3是否正常,导入import sqllite3是没有报错

[root@iZwz927xptl8kw8zqrmwvcZ Python-3.10.5]# python3
Python 3.10.5 (main, Jul 13 2022, 12:30:13) [GCC 8.5.0 20210514 (Red Hat 8.5.0-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.

>>> import sqlite3
>>> exit

sqlite3参考:https://www.cnblogs.com/laityguanguan/p/16186852.html

7)scrapyd 配置文件编辑

目录/usr/local/bin/python310/lib/python3.10/site-packages/scrapyd

找到default_scrapyd.conf,编辑

Scrapyd(centos 部署scrapy)安装介绍以及sqllite3配置

8)使用后动服务运行,切换到目录/usr/local/bin/python310/lib/python3.10/site-packages/scrapyd下启动

nohup scrapyd &

9)放开6800 端口,在外面访问,界面如下:

Scrapyd(centos 部署scrapy)安装介绍以及sqllite3配置

总结:之所以要修改tmp/Python-3.10.5/setup.py文件,是因为在安装sqlite3时,指定了安装路径./configure –prefix=/usr/local/bin/sqlite

Original: https://www.cnblogs.com/MrHSR/p/16435225.html
Author: 花阴偷移
Title: Scrapyd(centos 部署scrapy)安装介绍以及sqllite3配置

原创文章受到原创版权保护。转载请注明出处:https://www.johngo689.com/34865/

转载文章受原作者版权保护。转载请注明原作者出处!

(0)

大家都在看

最近整理资源【免费获取】:   👉 程序员最新必读书单  | 👏 互联网各方向面试题下载 | ✌️计算机核心资源汇总