Python 学习(二)
Python 开发环境配置
- 安装
requests
请求库,windows使用管理员打开cmd命令窗口,执行命令
C:\WINDOWS\system32>pip3 install requests
Collecting requestsUsing cached https://files.pythonhosted.org/packages/f1/ca/10332a30cb25b627192b4ea272c351bce3ca1091e541245cccbace6051d8/requests-2.20.0-py2.py3-none-any.whlCollecting chardet<3.1.0,>=3.0.2 (from requests)Using cached ...
Installing collected packages: chardet, certifi, idna, urllib3, requests
Successfully installed certifi-2018.10.15 chardet-3.0.4 idna-2.7 requests-2.20.0 urllib3-1.24
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
- 验证安装
C:\WINDOWS\system32>python
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:05:16) [MSC v.1915 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>>
看到如上,没有错误信息,表示安装成功
Selenium
的安装,自动话测试工具,使用pip
安装
C:\WINDOWS\system32>pip3 install selenium
Collecting seleniumDownloading https://files.pythonhosted.org/packages/b0/c9/52390baa8d6b65c3e3b89f522c3a0fcf58f2b4faf37893ef9d97cddde699/selenium-3.14.1-py2.py3-none-any.whl (902kB)100% |████████████████████████████████| 911kB 1.1MB/s
Requirement already satisfied: urllib3 in c:\program files (x86)\java\lib\site-packages(from selenium) (1.24)
Installing collected packages: selenium
Successfully installed selenium-3.14.1
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.C:\WINDOWS\system32>
- 验证安装
C:\WINDOWS\system32>python
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:05:16) [MSC v.1915 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import selenium
>>>
ChromeDriver
的安装,与 Selenium 配合使用- 下载
查看谷歌浏览器版本,下载对应的包,下载地址:
http://cdn.npm.taobao.org/dist/chromedriver/70.0.3538.67/chromedriver_win32.zip
- 解压,复制执行文件
chromedriver.exe
到Python的Scripts目录下,或者自己配置到环境变量中
C:\Program Files (x86)\Java\Scripts
- 验证安装
C:\WINDOWS\system32>chromedriver
Starting ChromeDriver 70.0.3538.67 (9ab0cfab84ded083718d3a4ff830726efd38869f) on port 9515
Only local connections are allowed.C:\Users\Dell>python
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:05:16) [MSC v.1915 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from selenium import webdriver
>>> browser = webdriver.Chrome()DevTools listening on ws://127.0.0.1:51172/devtools/browser/6ad9f3cc-308f-4a88-83a5-135b9443c95f
>>>
执行后如果弹出一个空白的Chrome浏览器,则表示安装成功。
- 下载
PhantomJS
,为无界面、可脚本编程的WebKit 浏览器引擎
下载地址:
https://bbuseruploads.s3.amazonaws.com/fd96ed93-2b32-46a7-9d2b-ecbc098851
6a/downloads/98d51451-997f-40e3-b9e6-a8e635dcdcb3/phantomjs-2.1.1-windows.zip?Signature=qC9Tz87sW7CEuE1ujLl9nf1qYPk%3D&Expires=1540901176&AWSAccessKeyId=AKIAIQWXW6WLXMB5QZAQ&versionId=null&response-content-disposition=attachment%3B%20filename%3D%22phantomjs-2.1.1-windows.zip%22原生支持多种web标准:DOM操作、CSS选择器、JSON、Canvas、SVG等
Selenium支持PhantomJS ,这样执行的时候则不会再弹出一个浏览器了,
运行效率高
- 安装
PhantomJS
,解压复制可执行文件phantomjs.exe
到Python的Script目录下 - 验证安装
C:\Users\Dell>phantomjs -v
2.1.1C:\Users\Dell>python
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:05:16) [MSC v.1915 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from selenium import webdriver
>>> browser = webdriver.PhantomJS()
>>> browser.get('http://www.baidu.com')
>>> print(browser.current_url)
https://www.baidu.com/
>>>
如上即代表我们访问了百度,然后将当前的地址打印处理,配置成功
aiohttp
异步web服务的库安装
C:\WINDOWS\system32>pip3 install aiohttp
Collecting aiohttpUsing cached https://files.pythonhosted.org/packages/7a/58/4476d96f35cc18a5133330f06c9ff3bc44fa64a9b6d15d2716efa6043b80/aiohttp-3.4.4-cp37-cp37m-win32.whl
Requirement already satisfied: chardet<4.0,>=2.0 in c:\program files (x86)\java\lib\site-packages(from aiohttp) (3.0.4)
...
Requirement already satisfied: idna>=2.0 in c:\program files (x86)\java\lib\site-packages(from yarl<2.0,>=1.0->aiohttp) (2.7)
Installing collected packages: attrs, async-timeout, multidict, yarl, aiohttpRunning setup.py install for yarl ... done
Successfully installed aiohttp-3.4.4 async-timeout-3.0.1 attrs-18.2.0 multidict-4.4.2 yarl-1.2.6
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.C:\WINDOWS\system32>
- 验证
C:\WINDOWS\system32>python
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:05:16) [MSC v.1915 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import aiohttp
>>>
- 安装字符编码检测库
cchardet
,加速DNS解析的库aiodns
Collecting cchardetDownloading https://files.pythonhosted.org/packages/dd/6a/6b54a269caa3518b6eaab3aec0fa4fa9eaa8019b4ae84a7c3ef66b1f9be0/cchardet-2.1.4-cp37-cp37m-win32.whl (89kB)100% |████████████████████████████████| 92kB 416kB/s
Installing collected packages: cchardet
Successfully installed cchardet-2.1.4
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
-
此处
aiodns
没有装成功 -
安装解析库
lxml
C:\WINDOWS\system32>pip3 install lxml
Collecting lxmlDownloading https://files.pythonhosted.org/packages/43/c7/e088bf0f4f81e6b366cc2de12939c559b588b9525ad76215d122e69151ed/lxml-4.2.5-cp37-cp37m-win32.whl (3.2MB)100% |████████████████████████████████| 3.2MB 512kB/s
Installing collected packages: lxml
Successfully installed lxml-4.2.5C:\WINDOWS\system32>
- 验证
C:\WINDOWS\system32>python
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:05:16) [MSC v.1915 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import lxml
>>>
Beautiful Soup
解析库安装
C:\WINDOWS\system32>pip3 install beautifulsoup4
Collecting beautifulsoup4Downloading https://files.pythonhosted.org/packages/21/0a/47fdf541c97fd9b6a610cb5fd518175308a7cc60569962e776ac52420387/beautifulsoup4-4.6.3-py3-none-any.whl (90kB)100% |████████████████████████████████| 92kB 475kB/s
Installing collected packages: beautifulsoup4
Successfully installed beautifulsoup4-4.6.3C:\WINDOWS\system32>
- 验证安装
C:\WINDOWS\system32>python
Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:05:16) [MSC v.1915 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import lxml
>>> from bs4 import BeautifulSoup
>>> soup = BeautifulSoup('<p>Hello</p>','lxml')
>>> print(soup.p.string)
Hello
>>>
pyquery
网页解析工具安装
C:\WINDOWS\system32>pip install pyquery
Collecting pyqueryDownloading https://files.pythonhosted.org/packages/09/c7/ce8c9c37ab8ff8337faad3335c088d60bed4a35a4bed33a64f0e64fbcf29/pyquery-1.4.0-py2.py3-none-any.whl
Requirement already satisfied: lxml>=2.1 in c:\program files (x86)\java\lib\site-packages
(from pyquery) (4.2.5)
Collecting cssselect>0.7.9 (from pyquery)Downloading https://files.pythonhosted.org/packages/7b/44/25b7283e50585f0b4156960691d951b05d061abf4a714078393e51929b30/cssselect-1.0.3-py2.py3-none-any.whl
Installing collected packages: cssselect, pyquery
Successfully installed cssselect-1.0.3 pyquery-1.4.0C:\WINDOWS\system32>
- 验证安装
>>> import pyquery
>>>
tesserocr
的安装,图形验证码识别库,此处没有安装成功
PyMysql
的安装,类似于数据库驱动包
C:\WINDOWS\system32>pip3 install pymysql
Collecting pymysqlDownloading https://files.pythonhosted.org/packages/a7/7d/682c4a7da195a678047c8f1c51bb7682aaedee1dca7547883c3993ca9282/PyMySQL-0.9.2-py2.py3-none-any.whl (47kB)100% |████████████████████████████████| 51kB 267kB/s
Collecting cryptography (from pymysql)
...
Installing collected packages: asn1crypto, pycparser, cffi, six, cryptography, pymysqlRunning setup.py install for pycparser ... done
Successfully installed asn1crypto-0.24.0 cffi-1.11.5 cryptography-2.3.1 pycparser-2.19pymysql-0.9.2 six-1.11.0C:\WINDOWS\system32>
- 验证安装
>>> import pyquery
>>> import pymysql
>>> pymysql.VERSION
(0, 9, 2, None)
>>>