You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

403 lines
11 KiB

4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
4 years ago
  1. Metadata-Version: 2.1
  2. Name: html5lib
  3. Version: 0.9999999
  4. Summary: HTML parser based on the WHATWG HTML specification
  5. Home-page: https://github.com/html5lib/html5lib-python
  6. Maintainer: James Graham
  7. Maintainer-email: james@hoppipolla.co.uk
  8. License: MIT License
  9. Platform: UNKNOWN
  10. Classifier: Development Status :: 5 - Production/Stable
  11. Classifier: Intended Audience :: Developers
  12. Classifier: License :: OSI Approved :: MIT License
  13. Classifier: Operating System :: OS Independent
  14. Classifier: Programming Language :: Python
  15. Classifier: Programming Language :: Python :: 2
  16. Classifier: Programming Language :: Python :: 2.6
  17. Classifier: Programming Language :: Python :: 2.7
  18. Classifier: Programming Language :: Python :: 3
  19. Classifier: Programming Language :: Python :: 3.2
  20. Classifier: Programming Language :: Python :: 3.3
  21. Classifier: Programming Language :: Python :: 3.4
  22. Classifier: Topic :: Software Development :: Libraries :: Python Modules
  23. Classifier: Topic :: Text Processing :: Markup :: HTML
  24. Requires-Dist: six
  25. html5lib
  26. ========
  27. .. image:: https://travis-ci.org/html5lib/html5lib-python.png?branch=master
  28. :target: https://travis-ci.org/html5lib/html5lib-python
  29. html5lib is a pure-python library for parsing HTML. It is designed to
  30. conform to the WHATWG HTML specification, as is implemented by all major
  31. web browsers.
  32. Usage
  33. -----
  34. Simple usage follows this pattern:
  35. .. code-block:: python
  36. import html5lib
  37. with open("mydocument.html", "rb") as f:
  38. document = html5lib.parse(f)
  39. or:
  40. .. code-block:: python
  41. import html5lib
  42. document = html5lib.parse("<p>Hello World!")
  43. By default, the ``document`` will be an ``xml.etree`` element instance.
  44. Whenever possible, html5lib chooses the accelerated ``ElementTree``
  45. implementation (i.e. ``xml.etree.cElementTree`` on Python 2.x).
  46. Two other tree types are supported: ``xml.dom.minidom`` and
  47. ``lxml.etree``. To use an alternative format, specify the name of
  48. a treebuilder:
  49. .. code-block:: python
  50. import html5lib
  51. with open("mydocument.html", "rb") as f:
  52. lxml_etree_document = html5lib.parse(f, treebuilder="lxml")
  53. When using with ``urllib2`` (Python 2), the charset from HTTP should be
  54. pass into html5lib as follows:
  55. .. code-block:: python
  56. from contextlib import closing
  57. from urllib2 import urlopen
  58. import html5lib
  59. with closing(urlopen("http://example.com/")) as f:
  60. document = html5lib.parse(f, encoding=f.info().getparam("charset"))
  61. When using with ``urllib.request`` (Python 3), the charset from HTTP
  62. should be pass into html5lib as follows:
  63. .. code-block:: python
  64. from urllib.request import urlopen
  65. import html5lib
  66. with urlopen("http://example.com/") as f:
  67. document = html5lib.parse(f, encoding=f.info().get_content_charset())
  68. To have more control over the parser, create a parser object explicitly.
  69. For instance, to make the parser raise exceptions on parse errors, use:
  70. .. code-block:: python
  71. import html5lib
  72. with open("mydocument.html", "rb") as f:
  73. parser = html5lib.HTMLParser(strict=True)
  74. document = parser.parse(f)
  75. When you're instantiating parser objects explicitly, pass a treebuilder
  76. class as the ``tree`` keyword argument to use an alternative document
  77. format:
  78. .. code-block:: python
  79. import html5lib
  80. parser = html5lib.HTMLParser(tree=html5lib.getTreeBuilder("dom"))
  81. minidom_document = parser.parse("<p>Hello World!")
  82. More documentation is available at http://html5lib.readthedocs.org/.
  83. Installation
  84. ------------
  85. html5lib works on CPython 2.6+, CPython 3.2+ and PyPy. To install it,
  86. use:
  87. .. code-block:: bash
  88. $ pip install html5lib
  89. Optional Dependencies
  90. ---------------------
  91. The following third-party libraries may be used for additional
  92. functionality:
  93. - ``datrie`` can be used to improve parsing performance (though in
  94. almost all cases the improvement is marginal);
  95. - ``lxml`` is supported as a tree format (for both building and
  96. walking) under CPython (but *not* PyPy where it is known to cause
  97. segfaults);
  98. - ``genshi`` has a treewalker (but not builder); and
  99. - ``charade`` can be used as a fallback when character encoding cannot
  100. be determined; ``chardet``, from which it was forked, can also be used
  101. on Python 2.
  102. - ``ordereddict`` can be used under Python 2.6
  103. (``collections.OrderedDict`` is used instead on later versions) to
  104. serialize attributes in alphabetical order.
  105. Bugs
  106. ----
  107. Please report any bugs on the `issue tracker
  108. <https://github.com/html5lib/html5lib-python/issues>`_.
  109. Tests
  110. -----
  111. Unit tests require the ``nose`` library and can be run using the
  112. ``nosetests`` command in the root directory; ``ordereddict`` is
  113. required under Python 2.6. All should pass.
  114. Test data are contained in a separate `html5lib-tests
  115. <https://github.com/html5lib/html5lib-tests>`_ repository and included
  116. as a submodule, thus for git checkouts they must be initialized::
  117. $ git submodule init
  118. $ git submodule update
  119. If you have all compatible Python implementations available on your
  120. system, you can run tests on all of them using the ``tox`` utility,
  121. which can be found on PyPI.
  122. Questions?
  123. ----------
  124. There's a mailing list available for support on Google Groups,
  125. `html5lib-discuss <http://groups.google.com/group/html5lib-discuss>`_,
  126. though you may get a quicker response asking on IRC in `#whatwg on
  127. irc.freenode.net <http://wiki.whatwg.org/wiki/IRC>`_.
  128. Change Log
  129. ----------
  130. 0.9999999/1.0b8
  131. ~~~~~~~~~~~~~~~
  132. Released on September 10, 2015
  133. * Fix #195: fix the sanitizer to drop broken URLs (it threw an
  134. exception between 0.9999 and 0.999999).
  135. 0.999999/1.0b7
  136. ~~~~~~~~~~~~~~
  137. Released on July 7, 2015
  138. * Fix #189: fix the sanitizer to allow relative URLs again (as it did
  139. prior to 0.9999/1.0b5).
  140. 0.99999/1.0b6
  141. ~~~~~~~~~~~~~
  142. Released on April 30, 2015
  143. * Fix #188: fix the sanitizer to not throw an exception when sanitizing
  144. bogus data URLs.
  145. 0.9999/1.0b5
  146. ~~~~~~~~~~~~
  147. Released on April 29, 2015
  148. * Fix #153: Sanitizer fails to treat some attributes as URLs. Despite how
  149. this sounds, this has no known security implications. No known version
  150. of IE (5.5 to current), Firefox (3 to current), Safari (6 to current),
  151. Chrome (1 to current), or Opera (12 to current) will run any script
  152. provided in these attributes.
  153. * Pass error message to the ParseError exception in strict parsing mode.
  154. * Allow data URIs in the sanitizer, with a whitelist of content-types.
  155. * Add support for Python implementations that don't support lone
  156. surrogates (read: Jython). Fixes #2.
  157. * Remove localization of error messages. This functionality was totally
  158. unused (and untested that everything was localizable), so we may as
  159. well follow numerous browsers in not supporting translating technical
  160. strings.
  161. * Expose treewalkers.pprint as a public API.
  162. * Add a documentEncoding property to HTML5Parser, fix #121.
  163. 0.999
  164. ~~~~~
  165. Released on December 23, 2013
  166. * Fix #127: add work-around for CPython issue #20007: .read(0) on
  167. http.client.HTTPResponse drops the rest of the content.
  168. * Fix #115: lxml treewalker can now deal with fragments containing, at
  169. their root level, text nodes with non-ASCII characters on Python 2.
  170. 0.99
  171. ~~~~
  172. Released on September 10, 2013
  173. * No library changes from 1.0b3; released as 0.99 as pip has changed
  174. behaviour from 1.4 to avoid installing pre-release versions per
  175. PEP 440.
  176. 1.0b3
  177. ~~~~~
  178. Released on July 24, 2013
  179. * Removed ``RecursiveTreeWalker`` from ``treewalkers._base``. Any
  180. implementation using it should be moved to
  181. ``NonRecursiveTreeWalker``, as everything bundled with html5lib has
  182. for years.
  183. * Fix #67 so that ``BufferedStream`` to correctly returns a bytes
  184. object, thereby fixing any case where html5lib is passed a
  185. non-seekable RawIOBase-like object.
  186. 1.0b2
  187. ~~~~~
  188. Released on June 27, 2013
  189. * Removed reordering of attributes within the serializer. There is now
  190. an ``alphabetical_attributes`` option which preserves the previous
  191. behaviour through a new filter. This allows attribute order to be
  192. preserved through html5lib if the tree builder preserves order.
  193. * Removed ``dom2sax`` from DOM treebuilders. It has been replaced by
  194. ``treeadapters.sax.to_sax`` which is generic and supports any
  195. treewalker; it also resolves all known bugs with ``dom2sax``.
  196. * Fix treewalker assertions on hitting bytes strings on
  197. Python 2. Previous to 1.0b1, treewalkers coped with mixed
  198. bytes/unicode data on Python 2; this reintroduces this prior
  199. behaviour on Python 2. Behaviour is unchanged on Python 3.
  200. 1.0b1
  201. ~~~~~
  202. Released on May 17, 2013
  203. * Implementation updated to implement the `HTML specification
  204. <http://www.whatwg.org/specs/web-apps/current-work/>`_ as of 5th May
  205. 2013 (`SVN <http://svn.whatwg.org/webapps/>`_ revision r7867).
  206. * Python 3.2+ supported in a single codebase using the ``six`` library.
  207. * Removed support for Python 2.5 and older.
  208. * Removed the deprecated Beautiful Soup 3 treebuilder.
  209. ``beautifulsoup4`` can use ``html5lib`` as a parser instead. Note that
  210. since it doesn't support namespaces, foreign content like SVG and
  211. MathML is parsed incorrectly.
  212. * Removed ``simpletree`` from the package. The default tree builder is
  213. now ``etree`` (using the ``xml.etree.cElementTree`` implementation if
  214. available, and ``xml.etree.ElementTree`` otherwise).
  215. * Removed the ``XHTMLSerializer`` as it never actually guaranteed its
  216. output was well-formed XML, and hence provided little of use.
  217. * Removed default DOM treebuilder, so ``html5lib.treebuilders.dom`` is no
  218. longer supported. ``html5lib.treebuilders.getTreeBuilder("dom")`` will
  219. return the default DOM treebuilder, which uses ``xml.dom.minidom``.
  220. * Optional heuristic character encoding detection now based on
  221. ``charade`` for Python 2.6 - 3.3 compatibility.
  222. * Optional ``Genshi`` treewalker support fixed.
  223. * Many bugfixes, including:
  224. * #33: null in attribute value breaks XML AttValue;
  225. * #4: nested, indirect descendant, <button> causes infinite loop;
  226. * `Google Code 215
  227. <http://code.google.com/p/html5lib/issues/detail?id=215>`_: Properly
  228. detect seekable streams;
  229. * `Google Code 206
  230. <http://code.google.com/p/html5lib/issues/detail?id=206>`_: add
  231. support for <video preload=...>, <audio preload=...>;
  232. * `Google Code 205
  233. <http://code.google.com/p/html5lib/issues/detail?id=205>`_: add
  234. support for <video poster=...>;
  235. * `Google Code 202
  236. <http://code.google.com/p/html5lib/issues/detail?id=202>`_: Unicode
  237. file breaks InputStream.
  238. * Source code is now mostly PEP 8 compliant.
  239. * Test harness has been improved and now depends on ``nose``.
  240. * Documentation updated and moved to http://html5lib.readthedocs.org/.
  241. 0.95
  242. ~~~~
  243. Released on February 11, 2012
  244. 0.90
  245. ~~~~
  246. Released on January 17, 2010
  247. 0.11.1
  248. ~~~~~~
  249. Released on June 12, 2008
  250. 0.11
  251. ~~~~
  252. Released on June 10, 2008
  253. 0.10
  254. ~~~~
  255. Released on October 7, 2007
  256. 0.9
  257. ~~~
  258. Released on March 11, 2007
  259. 0.2
  260. ~~~
  261. Released on January 8, 2007