You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

746 lines
19 KiB

4 years ago
  1. Metadata-Version: 2.1
  2. Name: voluptuous
  3. Version: 0.11.5
  4. Summary: # Voluptuous is a Python data validation library
  5. Home-page: https://github.com/alecthomas/voluptuous
  6. Author: Alec Thomas
  7. Author-email: alec@swapoff.org
  8. License: BSD
  9. Download-URL: https://pypi.python.org/pypi/voluptuous
  10. Platform: any
  11. Classifier: Development Status :: 5 - Production/Stable
  12. Classifier: Intended Audience :: Developers
  13. Classifier: License :: OSI Approved :: BSD License
  14. Classifier: Operating System :: OS Independent
  15. Classifier: Programming Language :: Python :: 2
  16. Classifier: Programming Language :: Python :: 2.7
  17. Classifier: Programming Language :: Python :: 3
  18. Classifier: Programming Language :: Python :: 3.6
  19. Classifier: Programming Language :: Python :: 3.7
  20. Description-Content-Type: text/markdown
  21. # Voluptuous is a Python data validation library
  22. [![Build Status](https://travis-ci.org/alecthomas/voluptuous.png)](https://travis-ci.org/alecthomas/voluptuous)
  23. [![Coverage Status](https://coveralls.io/repos/github/alecthomas/voluptuous/badge.svg?branch=master)](https://coveralls.io/github/alecthomas/voluptuous?branch=master) [![Gitter chat](https://badges.gitter.im/alecthomas.png)](https://gitter.im/alecthomas/Lobby)
  24. Voluptuous, *despite* the name, is a Python data validation library. It
  25. is primarily intended for validating data coming into Python as JSON,
  26. YAML, etc.
  27. It has three goals:
  28. 1. Simplicity.
  29. 2. Support for complex data structures.
  30. 3. Provide useful error messages.
  31. ## Contact
  32. Voluptuous now has a mailing list! Send a mail to
  33. [<voluptuous@librelist.com>](mailto:voluptuous@librelist.com) to subscribe. Instructions
  34. will follow.
  35. You can also contact me directly via [email](mailto:alec@swapoff.org) or
  36. [Twitter](https://twitter.com/alecthomas).
  37. To file a bug, create a [new issue](https://github.com/alecthomas/voluptuous/issues/new) on GitHub with a short example of how to replicate the issue.
  38. ## Documentation
  39. The documentation is provided [here](http://alecthomas.github.io/voluptuous/).
  40. ## Changelog
  41. See [CHANGELOG.md](https://github.com/alecthomas/voluptuous/blob/master/CHANGELOG.md).
  42. ## Show me an example
  43. Twitter's [user search API](https://dev.twitter.com/rest/reference/get/users/search) accepts
  44. query URLs like:
  45. ```
  46. $ curl 'https://api.twitter.com/1.1/users/search.json?q=python&per_page=20&page=1'
  47. ```
  48. To validate this we might use a schema like:
  49. ```pycon
  50. >>> from voluptuous import Schema
  51. >>> schema = Schema({
  52. ... 'q': str,
  53. ... 'per_page': int,
  54. ... 'page': int,
  55. ... })
  56. ```
  57. This schema very succinctly and roughly describes the data required by
  58. the API, and will work fine. But it has a few problems. Firstly, it
  59. doesn't fully express the constraints of the API. According to the API,
  60. `per_page` should be restricted to at most 20, defaulting to 5, for
  61. example. To describe the semantics of the API more accurately, our
  62. schema will need to be more thoroughly defined:
  63. ```pycon
  64. >>> from voluptuous import Required, All, Length, Range
  65. >>> schema = Schema({
  66. ... Required('q'): All(str, Length(min=1)),
  67. ... Required('per_page', default=5): All(int, Range(min=1, max=20)),
  68. ... 'page': All(int, Range(min=0)),
  69. ... })
  70. ```
  71. This schema fully enforces the interface defined in Twitter's
  72. documentation, and goes a little further for completeness.
  73. "q" is required:
  74. ```pycon
  75. >>> from voluptuous import MultipleInvalid, Invalid
  76. >>> try:
  77. ... schema({})
  78. ... raise AssertionError('MultipleInvalid not raised')
  79. ... except MultipleInvalid as e:
  80. ... exc = e
  81. >>> str(exc) == "required key not provided @ data['q']"
  82. True
  83. ```
  84. ...must be a string:
  85. ```pycon
  86. >>> try:
  87. ... schema({'q': 123})
  88. ... raise AssertionError('MultipleInvalid not raised')
  89. ... except MultipleInvalid as e:
  90. ... exc = e
  91. >>> str(exc) == "expected str for dictionary value @ data['q']"
  92. True
  93. ```
  94. ...and must be at least one character in length:
  95. ```pycon
  96. >>> try:
  97. ... schema({'q': ''})
  98. ... raise AssertionError('MultipleInvalid not raised')
  99. ... except MultipleInvalid as e:
  100. ... exc = e
  101. >>> str(exc) == "length of value must be at least 1 for dictionary value @ data['q']"
  102. True
  103. >>> schema({'q': '#topic'}) == {'q': '#topic', 'per_page': 5}
  104. True
  105. ```
  106. "per\_page" is a positive integer no greater than 20:
  107. ```pycon
  108. >>> try:
  109. ... schema({'q': '#topic', 'per_page': 900})
  110. ... raise AssertionError('MultipleInvalid not raised')
  111. ... except MultipleInvalid as e:
  112. ... exc = e
  113. >>> str(exc) == "value must be at most 20 for dictionary value @ data['per_page']"
  114. True
  115. >>> try:
  116. ... schema({'q': '#topic', 'per_page': -10})
  117. ... raise AssertionError('MultipleInvalid not raised')
  118. ... except MultipleInvalid as e:
  119. ... exc = e
  120. >>> str(exc) == "value must be at least 1 for dictionary value @ data['per_page']"
  121. True
  122. ```
  123. "page" is an integer \>= 0:
  124. ```pycon
  125. >>> try:
  126. ... schema({'q': '#topic', 'per_page': 'one'})
  127. ... raise AssertionError('MultipleInvalid not raised')
  128. ... except MultipleInvalid as e:
  129. ... exc = e
  130. >>> str(exc)
  131. "expected int for dictionary value @ data['per_page']"
  132. >>> schema({'q': '#topic', 'page': 1}) == {'q': '#topic', 'page': 1, 'per_page': 5}
  133. True
  134. ```
  135. ## Defining schemas
  136. Schemas are nested data structures consisting of dictionaries, lists,
  137. scalars and *validators*. Each node in the input schema is pattern
  138. matched against corresponding nodes in the input data.
  139. ### Literals
  140. Literals in the schema are matched using normal equality checks:
  141. ```pycon
  142. >>> schema = Schema(1)
  143. >>> schema(1)
  144. 1
  145. >>> schema = Schema('a string')
  146. >>> schema('a string')
  147. 'a string'
  148. ```
  149. ### Types
  150. Types in the schema are matched by checking if the corresponding value
  151. is an instance of the type:
  152. ```pycon
  153. >>> schema = Schema(int)
  154. >>> schema(1)
  155. 1
  156. >>> try:
  157. ... schema('one')
  158. ... raise AssertionError('MultipleInvalid not raised')
  159. ... except MultipleInvalid as e:
  160. ... exc = e
  161. >>> str(exc) == "expected int"
  162. True
  163. ```
  164. ### URL's
  165. URL's in the schema are matched by using `urlparse` library.
  166. ```pycon
  167. >>> from voluptuous import Url
  168. >>> schema = Schema(Url())
  169. >>> schema('http://w3.org')
  170. 'http://w3.org'
  171. >>> try:
  172. ... schema('one')
  173. ... raise AssertionError('MultipleInvalid not raised')
  174. ... except MultipleInvalid as e:
  175. ... exc = e
  176. >>> str(exc) == "expected a URL"
  177. True
  178. ```
  179. ### Lists
  180. Lists in the schema are treated as a set of valid values. Each element
  181. in the schema list is compared to each value in the input data:
  182. ```pycon
  183. >>> schema = Schema([1, 'a', 'string'])
  184. >>> schema([1])
  185. [1]
  186. >>> schema([1, 1, 1])
  187. [1, 1, 1]
  188. >>> schema(['a', 1, 'string', 1, 'string'])
  189. ['a', 1, 'string', 1, 'string']
  190. ```
  191. However, an empty list (`[]`) is treated as is. If you want to specify a list that can
  192. contain anything, specify it as `list`:
  193. ```pycon
  194. >>> schema = Schema([])
  195. >>> try:
  196. ... schema([1])
  197. ... raise AssertionError('MultipleInvalid not raised')
  198. ... except MultipleInvalid as e:
  199. ... exc = e
  200. >>> str(exc) == "not a valid value @ data[1]"
  201. True
  202. >>> schema([])
  203. []
  204. >>> schema = Schema(list)
  205. >>> schema([])
  206. []
  207. >>> schema([1, 2])
  208. [1, 2]
  209. ```
  210. ### Sets and frozensets
  211. Sets and frozensets are treated as a set of valid values. Each element
  212. in the schema set is compared to each value in the input data:
  213. ```pycon
  214. >>> schema = Schema({42})
  215. >>> schema({42}) == {42}
  216. True
  217. >>> try:
  218. ... schema({43})
  219. ... raise AssertionError('MultipleInvalid not raised')
  220. ... except MultipleInvalid as e:
  221. ... exc = e
  222. >>> str(exc) == "invalid value in set"
  223. True
  224. >>> schema = Schema({int})
  225. >>> schema({1, 2, 3}) == {1, 2, 3}
  226. True
  227. >>> schema = Schema({int, str})
  228. >>> schema({1, 2, 'abc'}) == {1, 2, 'abc'}
  229. True
  230. >>> schema = Schema(frozenset([int]))
  231. >>> try:
  232. ... schema({3})
  233. ... raise AssertionError('Invalid not raised')
  234. ... except Invalid as e:
  235. ... exc = e
  236. >>> str(exc) == 'expected a frozenset'
  237. True
  238. ```
  239. However, an empty set (`set()`) is treated as is. If you want to specify a set
  240. that can contain anything, specify it as `set`:
  241. ```pycon
  242. >>> schema = Schema(set())
  243. >>> try:
  244. ... schema({1})
  245. ... raise AssertionError('MultipleInvalid not raised')
  246. ... except MultipleInvalid as e:
  247. ... exc = e
  248. >>> str(exc) == "invalid value in set"
  249. True
  250. >>> schema(set()) == set()
  251. True
  252. >>> schema = Schema(set)
  253. >>> schema({1, 2}) == {1, 2}
  254. True
  255. ```
  256. ### Validation functions
  257. Validators are simple callables that raise an `Invalid` exception when
  258. they encounter invalid data. The criteria for determining validity is
  259. entirely up to the implementation; it may check that a value is a valid
  260. username with `pwd.getpwnam()`, it may check that a value is of a
  261. specific type, and so on.
  262. The simplest kind of validator is a Python function that raises
  263. ValueError when its argument is invalid. Conveniently, many builtin
  264. Python functions have this property. Here's an example of a date
  265. validator:
  266. ```pycon
  267. >>> from datetime import datetime
  268. >>> def Date(fmt='%Y-%m-%d'):
  269. ... return lambda v: datetime.strptime(v, fmt)
  270. ```
  271. ```pycon
  272. >>> schema = Schema(Date())
  273. >>> schema('2013-03-03')
  274. datetime.datetime(2013, 3, 3, 0, 0)
  275. >>> try:
  276. ... schema('2013-03')
  277. ... raise AssertionError('MultipleInvalid not raised')
  278. ... except MultipleInvalid as e:
  279. ... exc = e
  280. >>> str(exc) == "not a valid value"
  281. True
  282. ```
  283. In addition to simply determining if a value is valid, validators may
  284. mutate the value into a valid form. An example of this is the
  285. `Coerce(type)` function, which returns a function that coerces its
  286. argument to the given type:
  287. ```python
  288. def Coerce(type, msg=None):
  289. """Coerce a value to a type.
  290. If the type constructor throws a ValueError, the value will be marked as
  291. Invalid.
  292. """
  293. def f(v):
  294. try:
  295. return type(v)
  296. except ValueError:
  297. raise Invalid(msg or ('expected %s' % type.__name__))
  298. return f
  299. ```
  300. This example also shows a common idiom where an optional human-readable
  301. message can be provided. This can vastly improve the usefulness of the
  302. resulting error messages.
  303. ### Dictionaries
  304. Each key-value pair in a schema dictionary is validated against each
  305. key-value pair in the corresponding data dictionary:
  306. ```pycon
  307. >>> schema = Schema({1: 'one', 2: 'two'})
  308. >>> schema({1: 'one'})
  309. {1: 'one'}
  310. ```
  311. #### Extra dictionary keys
  312. By default any additional keys in the data, not in the schema will
  313. trigger exceptions:
  314. ```pycon
  315. >>> schema = Schema({2: 3})
  316. >>> try:
  317. ... schema({1: 2, 2: 3})
  318. ... raise AssertionError('MultipleInvalid not raised')
  319. ... except MultipleInvalid as e:
  320. ... exc = e
  321. >>> str(exc) == "extra keys not allowed @ data[1]"
  322. True
  323. ```
  324. This behaviour can be altered on a per-schema basis. To allow
  325. additional keys use
  326. `Schema(..., extra=ALLOW_EXTRA)`:
  327. ```pycon
  328. >>> from voluptuous import ALLOW_EXTRA
  329. >>> schema = Schema({2: 3}, extra=ALLOW_EXTRA)
  330. >>> schema({1: 2, 2: 3})
  331. {1: 2, 2: 3}
  332. ```
  333. To remove additional keys use
  334. `Schema(..., extra=REMOVE_EXTRA)`:
  335. ```pycon
  336. >>> from voluptuous import REMOVE_EXTRA
  337. >>> schema = Schema({2: 3}, extra=REMOVE_EXTRA)
  338. >>> schema({1: 2, 2: 3})
  339. {2: 3}
  340. ```
  341. It can also be overridden per-dictionary by using the catch-all marker
  342. token `extra` as a key:
  343. ```pycon
  344. >>> from voluptuous import Extra
  345. >>> schema = Schema({1: {Extra: object}})
  346. >>> schema({1: {'foo': 'bar'}})
  347. {1: {'foo': 'bar'}}
  348. ```
  349. #### Required dictionary keys
  350. By default, keys in the schema are not required to be in the data:
  351. ```pycon
  352. >>> schema = Schema({1: 2, 3: 4})
  353. >>> schema({3: 4})
  354. {3: 4}
  355. ```
  356. Similarly to how extra\_ keys work, this behaviour can be overridden
  357. per-schema:
  358. ```pycon
  359. >>> schema = Schema({1: 2, 3: 4}, required=True)
  360. >>> try:
  361. ... schema({3: 4})
  362. ... raise AssertionError('MultipleInvalid not raised')
  363. ... except MultipleInvalid as e:
  364. ... exc = e
  365. >>> str(exc) == "required key not provided @ data[1]"
  366. True
  367. ```
  368. And per-key, with the marker token `Required(key)`:
  369. ```pycon
  370. >>> schema = Schema({Required(1): 2, 3: 4})
  371. >>> try:
  372. ... schema({3: 4})
  373. ... raise AssertionError('MultipleInvalid not raised')
  374. ... except MultipleInvalid as e:
  375. ... exc = e
  376. >>> str(exc) == "required key not provided @ data[1]"
  377. True
  378. >>> schema({1: 2})
  379. {1: 2}
  380. ```
  381. #### Optional dictionary keys
  382. If a schema has `required=True`, keys may be individually marked as
  383. optional using the marker token `Optional(key)`:
  384. ```pycon
  385. >>> from voluptuous import Optional
  386. >>> schema = Schema({1: 2, Optional(3): 4}, required=True)
  387. >>> try:
  388. ... schema({})
  389. ... raise AssertionError('MultipleInvalid not raised')
  390. ... except MultipleInvalid as e:
  391. ... exc = e
  392. >>> str(exc) == "required key not provided @ data[1]"
  393. True
  394. >>> schema({1: 2})
  395. {1: 2}
  396. >>> try:
  397. ... schema({1: 2, 4: 5})
  398. ... raise AssertionError('MultipleInvalid not raised')
  399. ... except MultipleInvalid as e:
  400. ... exc = e
  401. >>> str(exc) == "extra keys not allowed @ data[4]"
  402. True
  403. ```
  404. ```pycon
  405. >>> schema({1: 2, 3: 4})
  406. {1: 2, 3: 4}
  407. ```
  408. ### Recursive / nested schema
  409. You can use `voluptuous.Self` to define a nested schema:
  410. ```pycon
  411. >>> from voluptuous import Schema, Self
  412. >>> recursive = Schema({"more": Self, "value": int})
  413. >>> recursive({"more": {"value": 42}, "value": 41}) == {'more': {'value': 42}, 'value': 41}
  414. True
  415. ```
  416. ### Extending an existing Schema
  417. Often it comes handy to have a base `Schema` that is extended with more
  418. requirements. In that case you can use `Schema.extend` to create a new
  419. `Schema`:
  420. ```pycon
  421. >>> from voluptuous import Schema
  422. >>> person = Schema({'name': str})
  423. >>> person_with_age = person.extend({'age': int})
  424. >>> sorted(list(person_with_age.schema.keys()))
  425. ['age', 'name']
  426. ```
  427. The original `Schema` remains unchanged.
  428. ### Objects
  429. Each key-value pair in a schema dictionary is validated against each
  430. attribute-value pair in the corresponding object:
  431. ```pycon
  432. >>> from voluptuous import Object
  433. >>> class Structure(object):
  434. ... def __init__(self, q=None):
  435. ... self.q = q
  436. ... def __repr__(self):
  437. ... return '<Structure(q={0.q!r})>'.format(self)
  438. ...
  439. >>> schema = Schema(Object({'q': 'one'}, cls=Structure))
  440. >>> schema(Structure(q='one'))
  441. <Structure(q='one')>
  442. ```
  443. ### Allow None values
  444. To allow value to be None as well, use Any:
  445. ```pycon
  446. >>> from voluptuous import Any
  447. >>> schema = Schema(Any(None, int))
  448. >>> schema(None)
  449. >>> schema(5)
  450. 5
  451. ```
  452. ## Error reporting
  453. Validators must throw an `Invalid` exception if invalid data is passed
  454. to them. All other exceptions are treated as errors in the validator and
  455. will not be caught.
  456. Each `Invalid` exception has an associated `path` attribute representing
  457. the path in the data structure to our currently validating value, as well
  458. as an `error_message` attribute that contains the message of the original
  459. exception. This is especially useful when you want to catch `Invalid`
  460. exceptions and give some feedback to the user, for instance in the context of
  461. an HTTP API.
  462. ```pycon
  463. >>> def validate_email(email):
  464. ... """Validate email."""
  465. ... if not "@" in email:
  466. ... raise Invalid("This email is invalid.")
  467. ... return email
  468. >>> schema = Schema({"email": validate_email})
  469. >>> exc = None
  470. >>> try:
  471. ... schema({"email": "whatever"})
  472. ... except MultipleInvalid as e:
  473. ... exc = e
  474. >>> str(exc)
  475. "This email is invalid. for dictionary value @ data['email']"
  476. >>> exc.path
  477. ['email']
  478. >>> exc.msg
  479. 'This email is invalid.'
  480. >>> exc.error_message
  481. 'This email is invalid.'
  482. ```
  483. The `path` attribute is used during error reporting, but also during matching
  484. to determine whether an error should be reported to the user or if the next
  485. match should be attempted. This is determined by comparing the depth of the
  486. path where the check is, to the depth of the path where the error occurred. If
  487. the error is more than one level deeper, it is reported.
  488. The upshot of this is that *matching is depth-first and fail-fast*.
  489. To illustrate this, here is an example schema:
  490. ```pycon
  491. >>> schema = Schema([[2, 3], 6])
  492. ```
  493. Each value in the top-level list is matched depth-first in-order. Given
  494. input data of `[[6]]`, the inner list will match the first element of
  495. the schema, but the literal `6` will not match any of the elements of
  496. that list. This error will be reported back to the user immediately. No
  497. backtracking is attempted:
  498. ```pycon
  499. >>> try:
  500. ... schema([[6]])
  501. ... raise AssertionError('MultipleInvalid not raised')
  502. ... except MultipleInvalid as e:
  503. ... exc = e
  504. >>> str(exc) == "not a valid value @ data[0][0]"
  505. True
  506. ```
  507. If we pass the data `[6]`, the `6` is not a list type and so will not
  508. recurse into the first element of the schema. Matching will continue on
  509. to the second element in the schema, and succeed:
  510. ```pycon
  511. >>> schema([6])
  512. [6]
  513. ```
  514. ## Multi-field validation
  515. Validation rules that involve multiple fields can be implemented as
  516. custom validators. It's recommended to use `All()` to do a two-pass
  517. validation - the first pass checking the basic structure of the data,
  518. and only after that, the second pass applying your cross-field
  519. validator:
  520. ```python
  521. def passwords_must_match(passwords):
  522. if passwords['password'] != passwords['password_again']:
  523. raise Invalid('passwords must match')
  524. return passwords
  525. s=Schema(All(
  526. # First "pass" for field types
  527. {'password':str, 'password_again':str},
  528. # Follow up the first "pass" with your multi-field rules
  529. passwords_must_match
  530. ))
  531. # valid
  532. s({'password':'123', 'password_again':'123'})
  533. # raises MultipleInvalid: passwords must match
  534. s({'password':'123', 'password_again':'and now for something completely different'})
  535. ```
  536. With this structure, your multi-field validator will run with
  537. pre-validated data from the first "pass" and so will not have to do
  538. its own type checking on its inputs.
  539. The flipside is that if the first "pass" of validation fails, your
  540. cross-field validator will not run:
  541. ```
  542. # raises Invalid because password_again is not a string
  543. # passwords_must_match() will not run because first-pass validation already failed
  544. s({'password':'123', 'password_again': 1337})
  545. ```
  546. ## Running tests.
  547. Voluptuous is using nosetests:
  548. $ nosetests
  549. ## Why use Voluptuous over another validation library?
  550. **Validators are simple callables**
  551. : No need to subclass anything, just use a function.
  552. **Errors are simple exceptions.**
  553. : A validator can just `raise Invalid(msg)` and expect the user to get
  554. useful messages.
  555. **Schemas are basic Python data structures.**
  556. : Should your data be a dictionary of integer keys to strings?
  557. `{int: str}` does what you expect. List of integers, floats or
  558. strings? `[int, float, str]`.
  559. **Designed from the ground up for validating more than just forms.**
  560. : Nested data structures are treated in the same way as any other
  561. type. Need a list of dictionaries? `[{}]`
  562. **Consistency.**
  563. : Types in the schema are checked as types. Values are compared as
  564. values. Callables are called to validate. Simple.
  565. ## Other libraries and inspirations
  566. Voluptuous is heavily inspired by
  567. [Validino](http://code.google.com/p/validino/), and to a lesser extent,
  568. [jsonvalidator](http://code.google.com/p/jsonvalidator/) and
  569. [json\_schema](http://blog.sendapatch.se/category/json_schema.html).
  570. [pytest-voluptuous](https://github.com/F-Secure/pytest-voluptuous) is a
  571. [pytest](https://github.com/pytest-dev/pytest) plugin that helps in
  572. using voluptuous validators in `assert`s.
  573. I greatly prefer the light-weight style promoted by these libraries to
  574. the complexity of libraries like FormEncode.