You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

530 lines
15 KiB

4 years ago
  1. Metadata-Version: 2.0
  2. Name: html5lib
  3. Version: 1.0.1
  4. Summary: HTML parser based on the WHATWG HTML specification
  5. Home-page: https://github.com/html5lib/html5lib-python
  6. Author: James Graham
  7. Author-email: james@hoppipolla.co.uk
  8. License: MIT License
  9. Description-Content-Type: UNKNOWN
  10. Platform: UNKNOWN
  11. Classifier: Development Status :: 5 - Production/Stable
  12. Classifier: Intended Audience :: Developers
  13. Classifier: License :: OSI Approved :: MIT License
  14. Classifier: Operating System :: OS Independent
  15. Classifier: Programming Language :: Python
  16. Classifier: Programming Language :: Python :: 2
  17. Classifier: Programming Language :: Python :: 2.7
  18. Classifier: Programming Language :: Python :: 3
  19. Classifier: Programming Language :: Python :: 3.3
  20. Classifier: Programming Language :: Python :: 3.4
  21. Classifier: Programming Language :: Python :: 3.5
  22. Classifier: Programming Language :: Python :: 3.6
  23. Classifier: Topic :: Software Development :: Libraries :: Python Modules
  24. Classifier: Topic :: Text Processing :: Markup :: HTML
  25. Requires-Dist: six (>=1.9)
  26. Requires-Dist: webencodings
  27. Provides-Extra: all
  28. Requires-Dist: genshi; extra == 'all'
  29. Requires-Dist: chardet (>=2.2); extra == 'all'
  30. Provides-Extra: all
  31. Requires-Dist: datrie; platform_python_implementation == 'CPython' and extra == 'all'
  32. Requires-Dist: lxml; platform_python_implementation == 'CPython' and extra == 'all'
  33. Provides-Extra: chardet
  34. Requires-Dist: chardet (>=2.2); extra == 'chardet'
  35. Provides-Extra: datrie
  36. Requires-Dist: datrie; platform_python_implementation == 'CPython' and extra == 'datrie'
  37. Provides-Extra: genshi
  38. Requires-Dist: genshi; extra == 'genshi'
  39. Provides-Extra: lxml
  40. Requires-Dist: lxml; platform_python_implementation == 'CPython' and extra == 'lxml'
  41. html5lib
  42. ========
  43. .. image:: https://travis-ci.org/html5lib/html5lib-python.png?branch=master
  44. :target: https://travis-ci.org/html5lib/html5lib-python
  45. html5lib is a pure-python library for parsing HTML. It is designed to
  46. conform to the WHATWG HTML specification, as is implemented by all major
  47. web browsers.
  48. Usage
  49. -----
  50. Simple usage follows this pattern:
  51. .. code-block:: python
  52. import html5lib
  53. with open("mydocument.html", "rb") as f:
  54. document = html5lib.parse(f)
  55. or:
  56. .. code-block:: python
  57. import html5lib
  58. document = html5lib.parse("<p>Hello World!")
  59. By default, the ``document`` will be an ``xml.etree`` element instance.
  60. Whenever possible, html5lib chooses the accelerated ``ElementTree``
  61. implementation (i.e. ``xml.etree.cElementTree`` on Python 2.x).
  62. Two other tree types are supported: ``xml.dom.minidom`` and
  63. ``lxml.etree``. To use an alternative format, specify the name of
  64. a treebuilder:
  65. .. code-block:: python
  66. import html5lib
  67. with open("mydocument.html", "rb") as f:
  68. lxml_etree_document = html5lib.parse(f, treebuilder="lxml")
  69. When using with ``urllib2`` (Python 2), the charset from HTTP should be
  70. pass into html5lib as follows:
  71. .. code-block:: python
  72. from contextlib import closing
  73. from urllib2 import urlopen
  74. import html5lib
  75. with closing(urlopen("http://example.com/")) as f:
  76. document = html5lib.parse(f, transport_encoding=f.info().getparam("charset"))
  77. When using with ``urllib.request`` (Python 3), the charset from HTTP
  78. should be pass into html5lib as follows:
  79. .. code-block:: python
  80. from urllib.request import urlopen
  81. import html5lib
  82. with urlopen("http://example.com/") as f:
  83. document = html5lib.parse(f, transport_encoding=f.info().get_content_charset())
  84. To have more control over the parser, create a parser object explicitly.
  85. For instance, to make the parser raise exceptions on parse errors, use:
  86. .. code-block:: python
  87. import html5lib
  88. with open("mydocument.html", "rb") as f:
  89. parser = html5lib.HTMLParser(strict=True)
  90. document = parser.parse(f)
  91. When you're instantiating parser objects explicitly, pass a treebuilder
  92. class as the ``tree`` keyword argument to use an alternative document
  93. format:
  94. .. code-block:: python
  95. import html5lib
  96. parser = html5lib.HTMLParser(tree=html5lib.getTreeBuilder("dom"))
  97. minidom_document = parser.parse("<p>Hello World!")
  98. More documentation is available at https://html5lib.readthedocs.io/.
  99. Installation
  100. ------------
  101. html5lib works on CPython 2.7+, CPython 3.3+ and PyPy. To install it,
  102. use:
  103. .. code-block:: bash
  104. $ pip install html5lib
  105. Optional Dependencies
  106. ---------------------
  107. The following third-party libraries may be used for additional
  108. functionality:
  109. - ``datrie`` can be used under CPython to improve parsing performance
  110. (though in almost all cases the improvement is marginal);
  111. - ``lxml`` is supported as a tree format (for both building and
  112. walking) under CPython (but *not* PyPy where it is known to cause
  113. segfaults);
  114. - ``genshi`` has a treewalker (but not builder); and
  115. - ``chardet`` can be used as a fallback when character encoding cannot
  116. be determined.
  117. Bugs
  118. ----
  119. Please report any bugs on the `issue tracker
  120. <https://github.com/html5lib/html5lib-python/issues>`_.
  121. Tests
  122. -----
  123. Unit tests require the ``pytest`` and ``mock`` libraries and can be
  124. run using the ``py.test`` command in the root directory.
  125. Test data are contained in a separate `html5lib-tests
  126. <https://github.com/html5lib/html5lib-tests>`_ repository and included
  127. as a submodule, thus for git checkouts they must be initialized::
  128. $ git submodule init
  129. $ git submodule update
  130. If you have all compatible Python implementations available on your
  131. system, you can run tests on all of them using the ``tox`` utility,
  132. which can be found on PyPI.
  133. Questions?
  134. ----------
  135. There's a mailing list available for support on Google Groups,
  136. `html5lib-discuss <http://groups.google.com/group/html5lib-discuss>`_,
  137. though you may get a quicker response asking on IRC in `#whatwg on
  138. irc.freenode.net <http://wiki.whatwg.org/wiki/IRC>`_.
  139. Change Log
  140. ----------
  141. 1.0.1
  142. ~~~~~
  143. Released on December 7, 2017
  144. Breaking changes:
  145. * Drop support for Python 2.6. (#330) (Thank you, Hugo, Will Kahn-Greene!)
  146. * Remove ``utils/spider.py`` (#353) (Thank you, Jon Dufresne!)
  147. Features:
  148. * Improve documentation. (#300, #307) (Thank you, Jon Dufresne, Tom Most,
  149. Will Kahn-Greene!)
  150. * Add iframe seamless boolean attribute. (Thank you, Ritwik Gupta!)
  151. * Add itemscope as a boolean attribute. (#194) (Thank you, Jonathan Vanasco!)
  152. * Support Python 3.6. (#333) (Thank you, Jon Dufresne!)
  153. * Add CI support for Windows using AppVeyor. (Thank you, John Vandenberg!)
  154. * Improve testing and CI and add code coverage (#323, #334), (Thank you, Jon
  155. Dufresne, John Vandenberg, Geoffrey Sneddon, Will Kahn-Greene!)
  156. * Semver-compliant version number.
  157. Bug fixes:
  158. * Add support for setuptools < 18.5 to support environment markers. (Thank you,
  159. John Vandenberg!)
  160. * Add explicit dependency for six >= 1.9. (Thank you, Eric Amorde!)
  161. * Fix regexes to work with Python 3.7 regex adjustments. (#318, #379) (Thank
  162. you, Benedikt Morbach, Ville Skyttä, Mark Vasilkov!)
  163. * Fix alphabeticalattributes filter namespace bug. (#324) (Thank you, Will
  164. Kahn-Greene!)
  165. * Include license file in generated wheel package. (#350) (Thank you, Jon
  166. Dufresne!)
  167. * Fix annotation-xml typo. (#339) (Thank you, Will Kahn-Greene!)
  168. * Allow uppercase hex chararcters in CSS colour check. (#377) (Thank you,
  169. Komal Dembla, Hugo!)
  170. 1.0
  171. ~~~
  172. Released and unreleased on December 7, 2017. Badly packaged release.
  173. 0.999999999/1.0b10
  174. ~~~~~~~~~~~~~~~~~~
  175. Released on July 15, 2016
  176. * Fix attribute order going to the tree builder to be document order
  177. instead of reverse document order(!).
  178. 0.99999999/1.0b9
  179. ~~~~~~~~~~~~~~~~
  180. Released on July 14, 2016
  181. * **Added ordereddict as a mandatory dependency on Python 2.6.**
  182. * Added ``lxml``, ``genshi``, ``datrie``, ``charade``, and ``all``
  183. extras that will do the right thing based on the specific
  184. interpreter implementation.
  185. * Now requires the ``mock`` package for the testsuite.
  186. * Cease supporting DATrie under PyPy.
  187. * **Remove PullDOM support, as this hasn't ever been properly
  188. tested, doesn't entirely work, and as far as I can tell is
  189. completely unused by anyone.**
  190. * Move testsuite to ``py.test``.
  191. * **Fix #124: move to webencodings for decoding the input byte stream;
  192. this makes html5lib compliant with the Encoding Standard, and
  193. introduces a required dependency on webencodings.**
  194. * **Cease supporting Python 3.2 (in both CPython and PyPy forms).**
  195. * **Fix comments containing double-dash with lxml 3.5 and above.**
  196. * **Use scripting disabled by default (as we don't implement
  197. scripting).**
  198. * **Fix #11, avoiding the XSS bug potentially caused by serializer
  199. allowing attribute values to be escaped out of in old browser versions,
  200. changing the quote_attr_values option on serializer to take one of
  201. three values, "always" (the old True value), "legacy" (the new option,
  202. and the new default), and "spec" (the old False value, and the old
  203. default).**
  204. * **Fix #72 by rewriting the sanitizer to apply only to treewalkers
  205. (instead of the tokenizer); as such, this will require amending all
  206. callers of it to use it via the treewalker API.**
  207. * **Drop support of charade, now that chardet is supported once more.**
  208. * **Replace the charset keyword argument on parse and related methods
  209. with a set of keyword arguments: override_encoding, transport_encoding,
  210. same_origin_parent_encoding, likely_encoding, and default_encoding.**
  211. * **Move filters._base, treebuilder._base, and treewalkers._base to .base
  212. to clarify their status as public.**
  213. * **Get rid of the sanitizer package. Merge sanitizer.sanitize into the
  214. sanitizer.htmlsanitizer module and move that to sanitizer. This means
  215. anyone who used sanitizer.sanitize or sanitizer.HTMLSanitizer needs no
  216. code changes.**
  217. * **Rename treewalkers.lxmletree to .etree_lxml and
  218. treewalkers.genshistream to .genshi to have a consistent API.**
  219. * Move a whole load of stuff (inputstream, ihatexml, trie, tokenizer,
  220. utils) to be underscore prefixed to clarify their status as private.
  221. 0.9999999/1.0b8
  222. ~~~~~~~~~~~~~~~
  223. Released on September 10, 2015
  224. * Fix #195: fix the sanitizer to drop broken URLs (it threw an
  225. exception between 0.9999 and 0.999999).
  226. 0.999999/1.0b7
  227. ~~~~~~~~~~~~~~
  228. Released on July 7, 2015
  229. * Fix #189: fix the sanitizer to allow relative URLs again (as it did
  230. prior to 0.9999/1.0b5).
  231. 0.99999/1.0b6
  232. ~~~~~~~~~~~~~
  233. Released on April 30, 2015
  234. * Fix #188: fix the sanitizer to not throw an exception when sanitizing
  235. bogus data URLs.
  236. 0.9999/1.0b5
  237. ~~~~~~~~~~~~
  238. Released on April 29, 2015
  239. * Fix #153: Sanitizer fails to treat some attributes as URLs. Despite how
  240. this sounds, this has no known security implications. No known version
  241. of IE (5.5 to current), Firefox (3 to current), Safari (6 to current),
  242. Chrome (1 to current), or Opera (12 to current) will run any script
  243. provided in these attributes.
  244. * Pass error message to the ParseError exception in strict parsing mode.
  245. * Allow data URIs in the sanitizer, with a whitelist of content-types.
  246. * Add support for Python implementations that don't support lone
  247. surrogates (read: Jython). Fixes #2.
  248. * Remove localization of error messages. This functionality was totally
  249. unused (and untested that everything was localizable), so we may as
  250. well follow numerous browsers in not supporting translating technical
  251. strings.
  252. * Expose treewalkers.pprint as a public API.
  253. * Add a documentEncoding property to HTML5Parser, fix #121.
  254. 0.999
  255. ~~~~~
  256. Released on December 23, 2013
  257. * Fix #127: add work-around for CPython issue #20007: .read(0) on
  258. http.client.HTTPResponse drops the rest of the content.
  259. * Fix #115: lxml treewalker can now deal with fragments containing, at
  260. their root level, text nodes with non-ASCII characters on Python 2.
  261. 0.99
  262. ~~~~
  263. Released on September 10, 2013
  264. * No library changes from 1.0b3; released as 0.99 as pip has changed
  265. behaviour from 1.4 to avoid installing pre-release versions per
  266. PEP 440.
  267. 1.0b3
  268. ~~~~~
  269. Released on July 24, 2013
  270. * Removed ``RecursiveTreeWalker`` from ``treewalkers._base``. Any
  271. implementation using it should be moved to
  272. ``NonRecursiveTreeWalker``, as everything bundled with html5lib has
  273. for years.
  274. * Fix #67 so that ``BufferedStream`` to correctly returns a bytes
  275. object, thereby fixing any case where html5lib is passed a
  276. non-seekable RawIOBase-like object.
  277. 1.0b2
  278. ~~~~~
  279. Released on June 27, 2013
  280. * Removed reordering of attributes within the serializer. There is now
  281. an ``alphabetical_attributes`` option which preserves the previous
  282. behaviour through a new filter. This allows attribute order to be
  283. preserved through html5lib if the tree builder preserves order.
  284. * Removed ``dom2sax`` from DOM treebuilders. It has been replaced by
  285. ``treeadapters.sax.to_sax`` which is generic and supports any
  286. treewalker; it also resolves all known bugs with ``dom2sax``.
  287. * Fix treewalker assertions on hitting bytes strings on
  288. Python 2. Previous to 1.0b1, treewalkers coped with mixed
  289. bytes/unicode data on Python 2; this reintroduces this prior
  290. behaviour on Python 2. Behaviour is unchanged on Python 3.
  291. 1.0b1
  292. ~~~~~
  293. Released on May 17, 2013
  294. * Implementation updated to implement the `HTML specification
  295. <http://www.whatwg.org/specs/web-apps/current-work/>`_ as of 5th May
  296. 2013 (`SVN <http://svn.whatwg.org/webapps/>`_ revision r7867).
  297. * Python 3.2+ supported in a single codebase using the ``six`` library.
  298. * Removed support for Python 2.5 and older.
  299. * Removed the deprecated Beautiful Soup 3 treebuilder.
  300. ``beautifulsoup4`` can use ``html5lib`` as a parser instead. Note that
  301. since it doesn't support namespaces, foreign content like SVG and
  302. MathML is parsed incorrectly.
  303. * Removed ``simpletree`` from the package. The default tree builder is
  304. now ``etree`` (using the ``xml.etree.cElementTree`` implementation if
  305. available, and ``xml.etree.ElementTree`` otherwise).
  306. * Removed the ``XHTMLSerializer`` as it never actually guaranteed its
  307. output was well-formed XML, and hence provided little of use.
  308. * Removed default DOM treebuilder, so ``html5lib.treebuilders.dom`` is no
  309. longer supported. ``html5lib.treebuilders.getTreeBuilder("dom")`` will
  310. return the default DOM treebuilder, which uses ``xml.dom.minidom``.
  311. * Optional heuristic character encoding detection now based on
  312. ``charade`` for Python 2.6 - 3.3 compatibility.
  313. * Optional ``Genshi`` treewalker support fixed.
  314. * Many bugfixes, including:
  315. * #33: null in attribute value breaks XML AttValue;
  316. * #4: nested, indirect descendant, <button> causes infinite loop;
  317. * `Google Code 215
  318. <http://code.google.com/p/html5lib/issues/detail?id=215>`_: Properly
  319. detect seekable streams;
  320. * `Google Code 206
  321. <http://code.google.com/p/html5lib/issues/detail?id=206>`_: add
  322. support for <video preload=...>, <audio preload=...>;
  323. * `Google Code 205
  324. <http://code.google.com/p/html5lib/issues/detail?id=205>`_: add
  325. support for <video poster=...>;
  326. * `Google Code 202
  327. <http://code.google.com/p/html5lib/issues/detail?id=202>`_: Unicode
  328. file breaks InputStream.
  329. * Source code is now mostly PEP 8 compliant.
  330. * Test harness has been improved and now depends on ``nose``.
  331. * Documentation updated and moved to https://html5lib.readthedocs.io/.
  332. 0.95
  333. ~~~~
  334. Released on February 11, 2012
  335. 0.90
  336. ~~~~
  337. Released on January 17, 2010
  338. 0.11.1
  339. ~~~~~~
  340. Released on June 12, 2008
  341. 0.11
  342. ~~~~
  343. Released on June 10, 2008
  344. 0.10
  345. ~~~~
  346. Released on October 7, 2007
  347. 0.9
  348. ~~~
  349. Released on March 11, 2007
  350. 0.2
  351. ~~~
  352. Released on January 8, 2007