You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

206 lines
4.9 KiB

4 years ago
  1. Metadata-Version: 2.0
  2. Name: queuelib
  3. Version: 1.5.0
  4. Summary: Collection of persistent (disk-based) queues
  5. Home-page: https://github.com/scrapy/queuelib
  6. Author: Scrapy project
  7. Author-email: info@scrapy.org
  8. License: BSD
  9. Description-Content-Type: UNKNOWN
  10. Platform: Any
  11. Classifier: Development Status :: 5 - Production/Stable
  12. Classifier: License :: OSI Approved :: BSD License
  13. Classifier: Operating System :: OS Independent
  14. Classifier: Programming Language :: Python
  15. Classifier: Programming Language :: Python :: 2
  16. Classifier: Programming Language :: Python :: 2.7
  17. Classifier: Programming Language :: Python :: 3
  18. Classifier: Programming Language :: Python :: 3.3
  19. Classifier: Programming Language :: Python :: 3.4
  20. Classifier: Programming Language :: Python :: Implementation :: CPython
  21. Classifier: Programming Language :: Python :: Implementation :: PyPy
  22. ========
  23. queuelib
  24. ========
  25. .. image:: https://secure.travis-ci.org/scrapy/queuelib.png?branch=master
  26. :target: http://travis-ci.org/scrapy/queuelib
  27. .. image:: https://img.shields.io/codecov/c/github/scrapy/queuelib/master.svg
  28. :target: http://codecov.io/github/scrapy/queuelib?branch=master
  29. :alt: Coverage report
  30. Queuelib is a collection of persistent (disk-based) queues for Python.
  31. Queuelib goals are speed and simplicity. It was originally part of the `Scrapy
  32. framework`_ and stripped out on its own library.
  33. Note: Queuelib isn't thread-safe.
  34. Requirements
  35. ============
  36. * Python 2.7 or Python 3.3
  37. * no external library requirements
  38. Installation
  39. ============
  40. You can install Queuelib either via the Python Package Index (PyPI) or from
  41. source.
  42. To install using pip::
  43. $ pip install queuelib
  44. To install using easy_install::
  45. $ easy_install queuelib
  46. If you have downloaded a source tarball you can install it by running the
  47. following (as root)::
  48. # python setup.py install
  49. FIFO/LIFO disk queues
  50. =====================
  51. Queuelib provides FIFO and LIFO queue implementations.
  52. Here is an example usage of the FIFO queue::
  53. >>> from queuelib import FifoDiskQueue
  54. >>> q = FifoDiskQueue("queuefile")
  55. >>> q.push(b'a')
  56. >>> q.push(b'b')
  57. >>> q.push(b'c')
  58. >>> q.pop()
  59. b'a'
  60. >>> q.close()
  61. >>> q = FifoDiskQueue("queuefile")
  62. >>> q.pop()
  63. b'b'
  64. >>> q.pop()
  65. b'c'
  66. >>> q.pop()
  67. >>>
  68. The LIFO queue is identical (API-wise), but importing ``LifoDiskQueue``
  69. instead.
  70. PriorityQueue
  71. =============
  72. A discrete-priority queue implemented by combining multiple FIFO/LIFO queues
  73. (one per priority).
  74. First, select the type of queue to be used per priority (FIFO or LIFO)::
  75. >>> from queuelib import FifoDiskQueue
  76. >>> qfactory = lambda priority: FifoDiskQueue('queue-dir-%s' % priority)
  77. Then instantiate the Priority Queue with it::
  78. >>> from queuelib import PriorityQueue
  79. >>> pq = PriorityQueue(qfactory)
  80. And use it::
  81. >>> pq.push(b'a', 3)
  82. >>> pq.push(b'b', 1)
  83. >>> pq.push(b'c', 2)
  84. >>> pq.push(b'd', 2)
  85. >>> pq.pop()
  86. b'b'
  87. >>> pq.pop()
  88. b'c'
  89. >>> pq.pop()
  90. b'd'
  91. >>> pq.pop()
  92. b'a'
  93. RoundRobinQueue
  94. ===============
  95. Has nearly the same interface and implementation as a Priority Queue except
  96. that each element must be pushed with a (mandatory) key. Popping from the
  97. queue cycles through the keys "round robin".
  98. Instantiate the Round Robin Queue similarly to the Priority Queue::
  99. >>> from queuelib import RoundRobinQueue
  100. >>> rr = RoundRobinQueue(qfactory)
  101. And use it::
  102. >>> rr.push(b'a', '1')
  103. >>> rr.push(b'b', '1')
  104. >>> rr.push(b'c', '2')
  105. >>> rr.push(b'd', '2')
  106. >>> rr.pop()
  107. b'a'
  108. >>> rr.pop()
  109. b'c'
  110. >>> rr.pop()
  111. b'b'
  112. >>> rr.pop()
  113. b'd'
  114. Mailing list
  115. ============
  116. Use the `scrapy-users`_ mailing list for questions about Queuelib.
  117. Bug tracker
  118. ===========
  119. If you have any suggestions, bug reports or annoyances please report them to
  120. our issue tracker at: http://github.com/scrapy/queuelib/issues/
  121. Contributing
  122. ============
  123. Development of Queuelib happens at GitHub: http://github.com/scrapy/queuelib
  124. You are highly encouraged to participate in the development. If you don't like
  125. GitHub (for some reason) you're welcome to send regular patches.
  126. All changes require tests to be merged.
  127. Tests
  128. =====
  129. Tests are located in `queuelib/tests` directory. They can be run using
  130. `nosetests`_ with the following command::
  131. nosetests
  132. The output should be something like the following::
  133. $ nosetests
  134. .............................................................................
  135. ----------------------------------------------------------------------
  136. Ran 77 tests in 0.145s
  137. OK
  138. License
  139. =======
  140. This software is licensed under the BSD License. See the LICENSE file in the
  141. top distribution directory for the full license text.
  142. Versioning
  143. ==========
  144. This software follows `Semantic Versioning`_
  145. .. _Scrapy framework: http://scrapy.org
  146. .. _scrapy-users: http://groups.google.com/group/scrapy-users
  147. .. _Semantic Versioning: http://semver.org/
  148. .. _nosetests: https://nose.readthedocs.org/en/latest/