python项目为了保持易读性,可维护性,做了代码格式规范,我大概总结了一下这些在格式化python代码中常用到的工具;
pep8
link:
pep8官方网站:https://www.python.org/dev/peps/pep-0008/
中文翻译版:https://alvinzhu.xyz/2017/10/07/python-pep-8/#top
pycodestyle
link:
https://pycodestyle.readthedocs.io/en/latest/
https://github.com/PyCQA/pycodestyle
简介
pycodestyle is a tool to check your Python code against some of the style conventions in PEP 8.
pycodestyle是一个检查python代码是否匹配pep8风格的工具
特点
- Plugin architecture: Adding new checks is easy(插件结构,安装起来很方便)
- Parseable output: Jump to error location in your editor.(输出易读,直接指出错误代码的位置)
- Small: Just one Python file, requires only stdlib. You can use just the pycodestyle.py file for this purpose.(安装文件小,就一个python脚本,外加依赖一个库stdlib)
- Comes with a comprehensive test suite.(代码检查很全面)
安装
1 | pip install pycodestyle |
使用
举例:
1 | $ pycodestyle --first optparse.py |
使用pycodestyle检查pep8语法的错误:
1 | $ pycodestyle --show-source --show-pep8 testsuite/E40.py |
统计状态码出现的次数:
1 | $ pycodestyle --statistics -qq Python-2.5/Lib |
官方的介绍更详细一点,实在是懒得翻译:
https://pycodestyle.readthedocs.io/en/latest/intro.html
flake8
link:
http://flake8.pycqa.org/en/latest/
https://pypi.org/project/flake8/
简介
Flake8 是由Python官方发布的一款辅助检测Python代码是否规范的工具,相对于目前热度比较高的Pylint来说,Flake8检查规则灵活,支持集成额外插件,扩展性强。Flake8是对下面三个工具的封装:
- PyFlakes:静态检查Python代码逻辑错误的工具。github:https://github.com/pycqa/pyflakes
- Pep8: 静态检查PEP8编码风格的工具。
- NedBatchelder’s McCabe :静态分析Python代码复杂度的工具。github: https://github.com/pycqa/mccabe
不光对以上三个工具的封装,Flake8还提供了扩展的开发接口。
安装
1 | pip install flake8 |
官方的安装解释:http://flake8.pycqa.org/en/latest/index.html#installation-guide
基本使用
1 | flake8 a.py |
一台机器上安装了多个版本的python时,指定python的版本使用flake8:
1 | 举例1:python3.5 -m flake8 |
使用flake8对一个项目做语法检查:
1 | flake8 my_project/ |
选项与配置文件
关于flake8 –help的哪些options的解释:http://flake8.pycqa.org/en/latest/user/options.html
flake8的options配置每次在使用的时候写到命令行有点麻烦,可以写到配置文件中,在一个project中支持的配置文件:setup.cfg, tox.ini, or .flake8.
并不是所有的命令行配置的options都可以写到配置文件里,
举个例子,命令行转换到配置文件中:
1 | flake8 --ignore D203 \ |
配置文件:
1 | [flake8] |
规范一点的话:
1 | [flake8] |
如果你愿意,也可以加上注释,这样可读性更好:
1 | [flake8] |
当你有很多配置的时候注释呢,作用还是很大的,例如:
1 | [flake8] |
状态码
Flake8 基础错误返回码一共有三类:
- E/W:PEP8 中的 error 和 warning。
- F***:通过 PyFlakes 检测出的 error,其实 PyFlakes 本身是不提供错误返回码的,flake8 对 pyflakes 返回的错误消息进行了分类。
- C9**:通过 McCabe 检测出的代码复杂度。
关于返回错误码的具体解释:
F开头的: http://flake8.pycqa.org/en/latest/user/error-codes.html
E和W开头的错:https://pycodestyle.readthedocs.io/en/latest/intro.html#error-codes
C开头的,代码的复杂度是怎么计算的:https://en.wikipedia.org/wiki/Cyclomatic_complexity
pyCodeStyle: http://pycodestyle.pycqa.org
插件
flake8的插件查找:
1 | pip search flake8 |
推荐使用的插件:
hacking
pep8-naming
flake8-chart
hacking
link:
https://pypi.python.org/pypi/hacking
https://docs.openstack.org/hacking/latest/user/index.html
简介
hacking就是flake8的一个插件,主要用来把代码规范为openstack style 的格式
hacking最初是基于nova的第一个commit,基于google python style的,后来逐渐增加了openstack style的一些规则,hacking主要实现了规则如下:
- Agree on a common style guide so reviews don’t get bogged down on style nit picks. (example: docstring guidelines)
- Make code written by many different authors easier to read by making the style more uniform. (example: unix vs windows newlines)
- Call out dangerous patterns and avoid them. (example: shadowing built-in or reserved words)
大致意思:
- 统一一个共同的风格指南,所以评论不用再挑选书写的风格。 (例如:docstring指南)
- 通过使样式更加统一,使许多不同作者编写的代码更容易阅读。 (例如:unix vs windows newlines)
- 指出危险模式并避免它们。 (例如:遮蔽内置或保留字)
安装
1 | pip install hacking |
hacking依赖于flake8,并且自己没有单独的命令,安装完后,执行flake8检查python脚本时,会有一堆H开头的错误,是hacking插件的;
使用
hacking的使用:
https://docs.openstack.org/hacking/latest/user/usage.html
什么是openstack style
yapf
link:
https://github.com/google/yapf
https://pypi.org/project/yapf/
简介
yapf(Yet Another Python Formatter)是Google开源的一个用来格式化Python代码的工具. 支持2种代码规范(PEP8和Google style)
安装
1 | pip install yapf |
返回状态码
还有返回状态码呢,当执行成功时返回0,否则返回非0数值
使用–diff 选项,当没有文件无变化是返回0,否则返回非0
这个特性可以用CI(持续集成)
.yapfignore
类似于.gitignore
配置
可配置在命令行,使用–style(可选择pep8或google),其他参数可使用{key:value}这种字典的格式写到命令行中
例如:
1 | --style='{based_on_style: chromium, indent_width: 4}' |
写到配置文件中举例:
1 | [yapf] |
配置的优先级选择:
默认选择pep8
- Specified on the command line
- In the [style] section of a .style.yapf file in either the current directory or one of its parent directories.
- In the [yapf] section of a setup.cfg file in either the current directory or one of its parent directories.
- In the ~/.config/yapf/style file in your home directory.
导出当前的配置文件
1 | yapf --style-help >yapf-style.config |
简单使用举例
原来的ugly code:
1 | x = { 'a':37,'b':42, |
经过yapf格式化后:
1 | x = {'a': 37, 'b': 42, 'c': 927} |
作为模块调用举例
yapf的可配置项(Knobs)
翻译成开关也可以
https://github.com/google/yapf#id10
使用yapf过程中可能会遇到的问题?
官方举了两个例子:
Why does YAPF destroy my awesome formatting?
Why Not Improve Existing Tools?
阔怕的细节(gory details):
看了一下看不懂,可怕,大致意思是用一个加权树算法来计算是否将一行分开,又是否合并两行,我把英文的贴到这,以后再看;
1 | Algorithm Design |
autopep8
link:
https://pypi.org/project/autopep8/0.8/
https://github.com/hhatto/autopep8
简介
Autopep8是一个将Python代码自动排版为PEP8风格的小工具。它使用pep8工具来决定代码中的哪部分需要被排版。Autopep8可以修复大部分pep8工具中报告的排版问题。
安装
1 | pip install autopep8 |
依赖于 pycodestyle
autopep8可以修复哪些状态码
(我管这些错误提示统称叫状态码)
第一种情况
autopep8可以修复一些pycodestyle的一些错误提示,具体的可以修复的状态码列表如下:
pycodestyle的错误格式提示码:
https://pycodestyle.readthedocs.io/en/latest/intro.html#error-codes
第二种情况
autopep8还可以修复一些不在pycodestyle格式范围内的错误:
- 纠正弃用的以及非惯用的python代码(通过 lib2to3)。这会让python 2.6 以及 python 2.7 的代码更加与 python3 兼容。(如果 W690 是enable的,这一项修复会被触发)
- 标准化具有多种行结束符的文件。
- 在类申明和它的第一个方法申明中间加一个空行。(由 E309 enable)
- 在类文档和它的第一个方法申明中间加一个空行。(由 E301 enable)
- 移除方法申明和它的文档之间的空行。 (由 E303 enable )
第三种情况
autopep8也会忽略一些pycodestyle检查出来的状态码:
E112( expected an indented block)/E113(unexpected indentation)
用于非注释的E112 / E113是破坏语法规则的不良缩进的报告。 这些不应该被修改。
E265 block comment should start with ‘# ‘
如果注释看起来像代码,则忽略注释哈希后的间距。 autopep8避免修改这些因为它们不是真正的注释。 如果您真的想摆脱pycodestyle警告,请考虑删除已注释掉的代码。 (这可以通过根除自动化。)
大致意思应该是说,E265报的错误是注释的一段代码,所以不把这段代码格式化;
autopep8使用
autopep8默认不修复E711/E712,因为怕我们自己重写了默认的eq方法,即把x == None 修改成 x is None
也不修改所有的W6警告(一些旧的书写格式告警):
–aggressive/-a选项可以提高代码修改错误码的等级,E712报的错误,就是在–aggressive等级是2 的时候修复(就是,x == True could be changed to either x or x is True, but autopep8 chooses the former)
–aggressive选项还会删除代码中的行末空格,要是想更加规范代码中的文档注释,可以使用 docformatter
可是你知道吗,我在此有一问,aggressive一共有几个等级?这文档也没有说明,或许只能从代码里找答案了…..
举例
一段不规范的代码
1 | import math, sys; |
autopep8格式化命令:
1 | autopep8 --in-place --aggressive --aggressive <filename> |
经过autopep8格式化后:
1 | import math |
其他常见命令行
指定autopep8只修改指定的状态码:
1 | autopep8 --select=E1,W1 <filename> |
只修改已被弃用的代码功能(W6)
1 | autopep8 --aggressive --select=W6 <filename> |
想要看详细的修改过程,可以使用选项-v:
1 | autopep8 -v <filename> |
autopep8配置文件
如果想把autopep8的选项放到配置文件中,把指定的选项修改到配置文件
autopep8默认识别的配置文件:
1 | linux:$HOME/.config/.pycodestyle |
与其他的配置也兼容的文件:
setup.cfg, tox.ini, .pep8 and .flake8
在pep8, pycodestyle, flake8中配置格式举例:
1 | [pycodestyle] |
使用autopep8自动化测试
可以写到项目的test/test_autopep8.py中,直接使用python test/test_autopep8.py来测试,也可以使用tox来测试,使用tox测试适用于测试是否适用多个python版本非常有用;
可以通过test / acid.py获得广谱测试。 此脚本针对Python代码运行autopep8,并检查代码修复的正确性和完整性。 它可以检查字节码是否保持相同。 test / acid_pypi.py使用acid.py来测试PyPI上最新发布的软件包。
什么是noqa:
no quality assurance(没有质量保证)
Adding # noqa to a line indicates that the linter (a program that automatically checks code quality) should not check this line. Any warnings that code may have generated will be ignored.
coverage
link:
https://coverage.readthedocs.io/en/v4.5.x/
https://pypi.org/project/coverage/
安装
1 | pip install coverage |
命令行
1 | run – Run a Python program and collect execution data. |
基本上就是跑一下覆盖率的测试然后把结果保存到html,xml等等
配置文件内容及格式:https://coverage.readthedocs.io/en/v4.5.x/config.html#config
coverage run
arg1与arg2是手动传入的参数
默认不指定任何参数时,是当前文件夹下所有的代码;
1 | $ coverage run my_program.py arg1 arg2 |
使用过程中可能用到的选项
- 使用-m选项也可以从模块导入要测试的函数或者类
1 | $ coverage run -m packagename.modulename arg1 arg2 |
- 使用–source <packages name/directories>
(<packages name/directories>使用逗号或换行符分隔)
使用source选项还可以使coverage.py报告未执行的文件,因为它可以在选项指定的包或目录中搜索尚未测试到的文件。这里所说的文件是指带有init.py可导入的模块 - –include
指定匹配的文件,patterns在此可使用正则,使用举例:/usr/*
如果source和include选项同时配置了,那么生效的是source选项;
也可以用在导出xml和html报告的输出命令中 - –omit
指定要忽略的文件,patterns使用同上
也可以用在导出xml和html报告的输出命令中 - –branch选项 https://coverage.readthedocs.io/en/v4.5.x/branch.html#branch
- –concurrency
multiprocessing, thread, greenlet, eventlet, or gevent
multiprocessing 不支持放在命令行 - –pylib
测试默认安装的包或模块 - –rcfile=FILE
指定配置文件,在配置文件中可配置的项比命令行选项更丰富
coverage warnnings
During execution, coverage.py may warn you about conditions it detects that could affect the measurement process. The possible warnings include:
“Trace function changed, measurement is likely wrong: XXX (trace-changed)”
Coverage measurement depends on a Python setting called the trace function. Other Python code in your product
might change that function, which will disrupt coverage.py’s measurement. This warning indicates that has happened.
The XXX in the message is the new trace function value, which might provide a clue to the cause.
“Module XXX has no Python source (module-not-python)”
You asked coverage.py to measure module XXX, but once it was imported, it turned out not to have a corresponding .py file. Without a .py file,
coverage.py can’t report on missing lines.
“Module XXX was never imported (module-not-imported)”
You asked coverage.py to measure module XXX, but it was never imported by your program.
“No data was collected (no-data-collected)”
Coverage.py ran your program, but didn’t measure any lines as executed. This could be because you asked to measure only modules that never ran, or for other reasons.
“Module XXX was previously imported, but not measured (module-not-measured)”
You asked coverage.py to measure module XXX, but it had already been imported when coverage started. This meant coverage.py couldn’t monitor its execution.
“–include is ignored because –source is set (include-ignored)”
Both –include and –source were specified while running code. Both are meant to focus measurement on a particular part of your source code, so –include is ignored in favor of –source.
配置这个warnning不可见
Individual warnings can be disabled with the disable_warnings configuration setting. To silence “No data was collected,” add this to your .coveragerc file:
[run]
disable_warnings = no-data-collected
输出结果:
默认是存放在.coverage文件中,可通过环境变量COVERAGE_FILE指定
如果想要保存多次结果可使用-a选项,将执行结果追加在上次执行结果的后面;
coverage erase 清除结果
1 | $ coverage erase |
coverage combine 合并当前目录下的coverage结果
1 | coverage combine |
合并指定名称,指定位置的结果
1 | $ coverage combine data1.dat windows_data_files/ |
如果不指定名称,将合并.coverage 和.coverage.xxx这种以.coverage为前缀的文件,举例:
1 | .coverage.machine1 |
更详细的功能参考:https://coverage.readthedocs.io/en/v4.5.x/cmd.html
converage report 输出报告
在输出报告时可能用到的选项:
–include/–omit 参考上文
-i/-ignore-errors
The -i or –ignore-errors switch tells coverage.py to ignore problems encountered trying to find source files to report on.
This can be useful if some files are missing, or if your Python execution is tricky enough that file names are synthesized without real source files.
–fail-under
If you provide a –fail-under value, the total percentage covered will be compared to that value. If it is less,
the command will exit with a status code of 2, indicating that the total coverage was less than your target.
This can be used as part of a pass/fail condition, for example in a continuous integration server. This option isn’t available for annotate.
-m shows the line numbers of missing statements, If you are using branch coverage, then branch statistics will be
reported in the Branch and BrPart (for Partial Branch) columns, the Missing column will detail the missed branches
–skip-covered
The –skip-covered switch will leave out any file with 100% coverage, letting you focus on the files that still need attention.
输出报告的命令:
1 | $ coverage report |
1 | $ coverage report -m |
HTML annotaion
生成命令:
The -d argument specifies an output directory, defaulting to “htmlcov”:
1 | $ coverage html -d coverage_html |
Text annotaion
XML reporting
The xml command writes coverage data to a “coverage.xml” file in a format compatible with Cobertura.
You can specify the name of the output file with the -o switch.
debug
The debug command shows internal information to help diagnose problems. If you are reporting a bug about coverage.py, including the output of this command can often help:
命令行使用:
1 | $ coverage debug sys > please_attach_to_bug_report.txt |
Three types of information are available:
config: show coverage’s configuration
sys: show system configuration,
data: show a summary of the collected coverage data