Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package python-deepdiff for openSUSE:Factory checked in at 2023-05-03 12:56:56 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-deepdiff (Old) and /work/SRC/openSUSE:Factory/.python-deepdiff.new.1533 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-deepdiff" Wed May 3 12:56:56 2023 rev:10 rq:1084088 version:6.3.0 Changes: -------- --- /work/SRC/openSUSE:Factory/python-deepdiff/python-deepdiff.changes 2023-02-21 15:36:39.564499877 +0100 +++ /work/SRC/openSUSE:Factory/.python-deepdiff.new.1533/python-deepdiff.changes 2023-05-03 12:57:14.439819587 +0200 @@ -1,0 +2,21 @@ +Tue May 2 15:55:35 UTC 2023 - Daniel Garcia <daniel.gar...@suse.com> + +- Update to 6.3.0: + - PrefixOrSuffixOperator: This operator will skip strings that are + suffix or prefix of each other. + - include_obj_callback and include_obj_callback_strict are added + by [HÃ¥vard Thom](https://github.com/havardthom). + - Fixed a corner case where numpy's np.float32 nans are not + ignored when using ignore_nan_equality by [Noam + Gottlieb](https://github.com/noamgot) + - orjson becomes optional again. + - Fix for ignore_type_in_groups with numeric values so it does not + report number changes when the number types are different. + +------------------------------------------------------------------- +Tue May 2 15:47:02 UTC 2023 - Daniel Garcia <daniel.gar...@suse.com> + +- Move python-orjson from Recommends to Requires, it's a requirement + right now in the latest release. + +------------------------------------------------------------------- Old: ---- deepdiff-6.2.3-gh.tar.gz New: ---- deepdiff-6.3.0-gh.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-deepdiff.spec ++++++ --- /var/tmp/diff_new_pack.unqXd9/_old 2023-05-03 12:57:14.943822548 +0200 +++ /var/tmp/diff_new_pack.unqXd9/_new 2023-05-03 12:57:14.947822571 +0200 @@ -18,12 +18,12 @@ %define skip_python2 1 Name: python-deepdiff -Version: 6.2.3 +Version: 6.3.0 Release: 0 Summary: Deep Difference and Search of any Python object/data License: MIT URL: https://github.com/seperman/deepdiff -Source: https://github.com/seperman/deepdiff/archive/v%{version}.tar.gz#/deepdiff-%{version}-gh.tar.gz +Source: https://github.com/seperman/deepdiff/archive/%{version}.tar.gz#/deepdiff-%{version}-gh.tar.gz BuildRequires: %{python_module PyYAML} BuildRequires: %{python_module click} BuildRequires: %{python_module jsonpickle} ++++++ deepdiff-6.2.3-gh.tar.gz -> deepdiff-6.3.0-gh.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/AUTHORS.md new/deepdiff-6.3.0/AUTHORS.md --- old/deepdiff-6.2.3/AUTHORS.md 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/AUTHORS.md 2023-03-17 19:27:19.000000000 +0100 @@ -50,3 +50,5 @@ - [Uwe Fladrich](https://github.com/uwefladrich) for fixing bug when diff'ing non-sequence iterables - [Michal Ozery-Flato](https://github.com/michalozeryflato) for setting equal_nan=ignore_nan_inequality in the call for np.array_equal - [martin-kokos](https://github.com/martin-kokos) for using Pytest's tmp_path fixture instead of /tmp/ +- HÃ¥vard Thom [havardthom](https://github.com/havardthom) for adding include_obj_callback and include_obj_callback_strict. +- [Noam Gottlieb](https://github.com/noamgot) for fixing a corner case where numpy's `np.float32` nans are not ignored when using `ignore_nan_equality`. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/CHANGELOG.md new/deepdiff-6.3.0/CHANGELOG.md --- old/deepdiff-6.2.3/CHANGELOG.md 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/CHANGELOG.md 2023-03-17 19:27:19.000000000 +0100 @@ -1,5 +1,11 @@ # DeepDiff Change log +- v6-3-0 + - `PrefixOrSuffixOperator`: This operator will skip strings that are suffix or prefix of each other. + - `include_obj_callback` and `include_obj_callback_strict` are added by [HÃ¥vard Thom](https://github.com/havardthom). + - Fixed a corner case where numpy's `np.float32` nans are not ignored when using `ignore_nan_equality` by [Noam Gottlieb](https://github.com/noamgot) + - `orjson` becomes optional again. + - Fix for `ignore_type_in_groups` with numeric values so it does not report number changes when the number types are different. - v6-2-3 - Switching to Orjson for serialization to improve the performance. - Setting `equal_nan=ignore_nan_inequality` in the call for `np.array_equal` diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/README.md new/deepdiff-6.3.0/README.md --- old/deepdiff-6.2.3/README.md 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/README.md 2023-03-17 19:27:19.000000000 +0100 @@ -1,4 +1,4 @@ -# DeepDiff v 6.2.3 +# DeepDiff v 6.3.0   @@ -6,37 +6,35 @@ [](https://github.com/seperman/deepdiff/actions) [](https://codecov.io/gh/seperman/deepdiff) -## DeepDiff Overview +## Modules -- DeepDiff: Deep Difference of dictionaries, iterables, strings and other objects. It will recursively look for all the changes. -- DeepSearch: Search for objects within other objects. -- DeepHash: Hash any object based on their content. +- [DeepDiff](https://zepworks.com/deepdiff/current/diff.html): Deep Difference of dictionaries, iterables, strings, and ANY other object. +- [DeepSearch](https://zepworks.com/deepdiff/current/dsearch.html): Search for objects within other objects. +- [DeepHash](https://zepworks.com/deepdiff/current/deephash.html): Hash any object based on their content. +- [Delta](https://zepworks.com/deepdiff/current/delta.html): Store the difference of objects and apply them to other objects. +- [Extract](https://zepworks.com/deepdiff/current/extract.html): Extract an item from a nested Python object using its path. +- [commandline](https://zepworks.com/deepdiff/current/commandline.html): Use DeepDiff from commandline. Tested on Python 3.7+ and PyPy3. -- **[Documentation](https://zepworks.com/deepdiff/6.2.3/)** +- **[Documentation](https://zepworks.com/deepdiff/6.3.0/)** ## What is new? Please check the [ChangeLog](CHANGELOG.md) file for the detailed information. -DeepDiff 6-2-0 - -- Major improvement in the diff report for lists when items are all hashable and the order of items is important. - -DeepDiff 6-1-0 +DeepDiff 6-3-0 -- DeepDiff.affected_paths can be used to get the list of all paths where a change, addition, or deletion was reported for. -- DeepDiff.affected_root_keys can be used to get the list of all paths where a change, addition, or deletion was reported for. -- Bugfix: ValueError when using Decimal 0.x #339 by [Enric Pou](https://github.com/epou) -- Serialization of UUID +- [`PrefixOrSuffixOperator`](https://zepworks.com/deepdiff/current/custom.html#prefix-or-suffix-operator-label): This operator will skip strings that are suffix or prefix of each other. +- [`include_obj_callback`](https://zepworks.com/deepdiff/current/ignore_types_or_values.html#include-obj-callback-label) and `include_obj_callback_strict` are added by [HÃ¥vard Thom](https://github.com/havardthom). +- Fixed a corner case where numpy's `np.float32` nans are not ignored when using `ignore_nan_equality` by [Noam Gottlieb](https://github.com/noamgot) +- `orjson` becomes optional again. +- Fix for `ignore_type_in_groups` with numeric values so it does not report number changes when the number types are different. -DeepDiff 6-0-0 +DeepDiff 6-2-0 -- [Exclude obj callback strict](https://github.com/seperman/deepdiff/pull/320/files) parameter is added to DeepDiff by Mikhail Khviyuzov [mskhviyu](https://github.com/mskhviyu). -- A fix for diffing using `iterable_compare_func` with nested objects by [dtorres-sf](https://github.com/dtorres-sf) who originally contributed this feature. +- Major improvement in the diff report for lists when items are all hashable and the order of items is important. -Note: There are no breaking changes in DeepDiff 6 compared to the latest DeepDiff 5 releases. ## Installation @@ -48,396 +46,18 @@ `pip install "deepdiff[cli]"` -### Importing - -```python ->>> from deepdiff import DeepDiff # For Deep Difference of 2 objects ->>> from deepdiff import grep, DeepSearch # For finding if item exists in an object ->>> from deepdiff import DeepHash # For hashing objects based on their contents -``` - -Note: if you want to use DeepDiff via commandline, make sure to run `pip install "deepdiff[cli]"`. Then you can access the commands via: - -- DeepDiff - - `$ deep diff --help` -- Delta - - `$ deep patch --help` -- grep - - `$ deep grep --help` -- extract - - `$ deep extract --help` - -# Deep Diff - -DeepDiff gets the difference of 2 objects. - -> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/6.2.3/diff.html) -> - The full documentation of all modules can be found on <https://zepworks.com/deepdiff/6.2.3/> -> - Tutorials and posts about DeepDiff can be found on <https://zepworks.com/tags/deepdiff/> - -## A few Examples - -> Note: This is just a brief overview of what DeepDiff can do. Please visit <https://zepworks.com/deepdiff/6.2.3/> for full documentation. - -### List difference ignoring order or duplicates - -```python ->>> t1 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 2, 3]}} ->>> t2 = {1:1, 2:2, 3:3, 4:{"a":"hello", "b":[1, 3, 2, 3]}} ->>> ddiff = DeepDiff(t1, t2, ignore_order=True) ->>> print (ddiff) -{} -``` - -### Report repetitions - -This flag ONLY works when ignoring order is enabled. - -```python -t1 = [1, 3, 1, 4] -t2 = [4, 4, 1] -ddiff = DeepDiff(t1, t2, ignore_order=True, report_repetition=True) -print(ddiff) -``` - -which will print you: - -```python -{'iterable_item_removed': {'root[1]': 3}, - 'repetition_change': {'root[0]': {'old_repeat': 2, - 'old_indexes': [0, 2], - 'new_indexes': [2], - 'value': 1, - 'new_repeat': 1}, - 'root[3]': {'old_repeat': 1, - 'old_indexes': [3], - 'new_indexes': [0, 1], - 'value': 4, - 'new_repeat': 2}}} -``` - -### Exclude certain types from comparison: - -```python ->>> l1 = logging.getLogger("test") ->>> l2 = logging.getLogger("test2") ->>> t1 = {"log": l1, 2: 1337} ->>> t2 = {"log": l2, 2: 1337} ->>> print(DeepDiff(t1, t2, exclude_types={logging.Logger})) -{} -``` - -### Exclude part of your object tree from comparison - -```python ->>> t1 = {"for life": "vegan", "ingredients": ["no meat", "no eggs", "no dairy"]} ->>> t2 = {"for life": "vegan", "ingredients": ["veggies", "tofu", "soy sauce"]} ->>> print (DeepDiff(t1, t2, exclude_paths={"root['ingredients']"})) -{} -``` - -### Exclude Regex Paths - - -You can also exclude using regular expressions by using `exclude_regex_paths` and pass a set or list of path regexes to exclude. The items in the list could be raw regex strings or compiled regex objects. - -```python ->>> t1 = [{'a': 1, 'b': 2}, {'c': 4, 'b': 5}] ->>> t2 = [{'a': 1, 'b': 3}, {'c': 4, 'b': 5}] ->>> print(DeepDiff(t1, t2, exclude_regex_paths={r"root\[\d+\]\['b'\]"})) -{} ->>> exclude_path = re.compile(r"root\[\d+\]\['b'\]") ->>> print(DeepDiff(t1, t2, exclude_regex_paths=[exclude_path])) -{} -``` - -### Significant Digits - -Digits **after** the decimal point. Internally it uses "{:.Xf}".format(Your Number) to compare numbers where X=significant_digits - -```python ->>> t1 = Decimal('1.52') ->>> t2 = Decimal('1.57') ->>> DeepDiff(t1, t2, significant_digits=0) -{} ->>> DeepDiff(t1, t2, significant_digits=1) -{'values_changed': {'root': {'old_value': Decimal('1.52'), 'new_value': Decimal('1.57')}}} -``` - -### Ignore Type Number - List that contains float and integer: - -```py ->>> from deepdiff import DeepDiff ->>> from pprint import pprint ->>> t1 = [1, 2, 3] ->>> t2 = [1.0, 2.0, 3.0] ->>> ddiff = DeepDiff(t1, t2) ->>> pprint(ddiff, indent=2) -{ 'type_changes': { 'root[0]': { 'new_type': <class 'float'>, - 'new_value': 1.0, - 'old_type': <class 'int'>, - 'old_value': 1}, - 'root[1]': { 'new_type': <class 'float'>, - 'new_value': 2.0, - 'old_type': <class 'int'>, - 'old_value': 2}, - 'root[2]': { 'new_type': <class 'float'>, - 'new_value': 3.0, - 'old_type': <class 'int'>, - 'old_value': 3}}} ->>> ddiff = DeepDiff(t1, t2, ignore_type_in_groups=[(int, float)]) -{} -``` - -## Views - -Starting with DeepDiff v 3, there are two different views into your diffed data: text view (original) and tree view (new). - -### Text View - -Text view is the original and currently the default view of DeepDiff. - -It is called text view because the results contain texts that represent the path to the data: - -Example of using the text view. - -```python ->>> from deepdiff import DeepDiff ->>> t1 = {1:1, 3:3, 4:4} ->>> t2 = {1:1, 3:3, 5:5, 6:6} ->>> ddiff = DeepDiff(t1, t2) ->>> print(ddiff) -{'dictionary_item_added': {'root[5]', 'root[6]'}, 'dictionary_item_removed': {'root[4]'}} -``` - -So for example `ddiff['dictionary_item_removed']` is a set if strings thus this is called the text view. - - The following examples are using the *default text view.* - The Tree View is introduced in DeepDiff v3 - and provides traversing capabilities through your diffed data and more! - Read more about the Tree View at the [tree view section](#tree-view) of this page. - - -### Tree View - -Starting the version v3 You can choose the view into the deepdiff results. -The tree view provides you with tree objects that you can traverse through to find the parents of the objects that are diffed and the actual objects that are being diffed. - - -#### Value of an item has changed (Tree View) - -```python ->>> from deepdiff import DeepDiff ->>> from pprint import pprint ->>> t1 = {1:1, 2:2, 3:3} ->>> t2 = {1:1, 2:4, 3:3} ->>> ddiff_verbose0 = DeepDiff(t1, t2, verbose_level=0, view='tree') ->>> ddiff_verbose0 -{'values_changed': {<root[2]>}} ->>> ->>> ddiff_verbose1 = DeepDiff(t1, t2, verbose_level=1, view='tree') ->>> ddiff_verbose1 -{'values_changed': {<root[2] t1:2, t2:4>}} ->>> set_of_values_changed = ddiff_verbose1['values_changed'] ->>> # since set_of_values_changed includes only one item in a set ->>> # in order to get that one item we can: ->>> (changed,) = set_of_values_changed ->>> changed # Another way to get this is to do: changed=list(set_of_values_changed)[0] -<root[2] t1:2, t2:4> ->>> changed.t1 -2 ->>> changed.t2 -4 ->>> # You can traverse through the tree, get to the parents! ->>> changed.up -<root t1:{1: 1, 2: 2,...}, t2:{1: 1, 2: 4,...}> -``` - -### Serialization - -In order to convert the DeepDiff object into a normal Python dictionary, use the to_dict() method. -Note that to_dict will use the text view even if you did the diff in tree view. - -Example: - -```python ->>> t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, 3]}} ->>> t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": "world\n\n\nEnd"}} ->>> ddiff = DeepDiff(t1, t2, view='tree') ->>> ddiff.to_dict() -{'type_changes': {"root[4]['b']": {'old_type': <class 'list'>, 'new_type': <class 'str'>, 'old_value': [1, 2, 3], 'new_value': 'world\n\n\nEnd'}}} -``` - -In order to do safe json serialization, use the to_json() method. - -Example: - -```python ->>> t1 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": [1, 2, 3]}} ->>> t2 = {1: 1, 2: 2, 3: 3, 4: {"a": "hello", "b": "world\n\n\nEnd"}} ->>> ddiff = DeepDiff(t1, t2, view='tree') ->>> ddiff.to_json() -'{"type_changes": {"root[4][\'b\']": {"old_type": "list", "new_type": "str", "old_value": [1, 2, 3], "new_value": "world\\n\\n\\nEnd"}}}' -``` - - -> - Please take a look at the [DeepDiff docs](https://zepworks.com/deepdiff/6.2.3/diff.html) -> - The full documentation can be found on <https://zepworks.com/deepdiff/6.2.3/> - - -# Deep Search - -DeepDiff comes with a utility to find the path to the item you are looking for. -It is called DeepSearch and it has a similar interface to DeepDiff. - -Let's say you have a huge nested object and want to see if any item with the word `somewhere` exists in it. -Just grep through your objects as you would in shell! - -```py -from deepdiff import grep -obj = {"long": "somewhere", "string": 2, 0: 0, "somewhere": "around"} -ds = obj | grep("somewhere") -print(ds) -``` - -Which will print: - -```py -{'matched_paths': {"root['somewhere']"}, - 'matched_values': {"root['long']"}} -``` - -And you can pass all the same kwargs as DeepSearch to grep too: - -```py ->>> obj | grep(item, verbose_level=2) -{'matched_paths': {"root['somewhere']": 'around'}, 'matched_values': {"root['long']": 'somewhere'}} -``` - -> - Please take a look at the [DeepSearch docs](https://zepworks.com/deepdiff/6.2.3/dsearch.html) -> - The full documentation can be found on <https://zepworks.com/deepdiff/6.2.3/> - -# Deep Hash -(New in v4-0-0) - -DeepHash is designed to give you hash of ANY python object based on its contents even if the object is not considered hashable! -DeepHash is supposed to be deterministic in order to make sure 2 objects that contain the same data, produce the same hash. - -> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/6.2.3/deephash.html) -> - The full documentation can be found on <https://zepworks.com/deepdiff/6.2.3/> - -Let's say you have a dictionary object. - -```py ->>> from deepdiff import DeepHash ->>> ->>> obj = {1: 2, 'a': 'b'} -``` - -If you try to hash it: - -```py ->>> hash(obj) -Traceback (most recent call last): - File "<stdin>", line 1, in <module> -TypeError: unhashable type: 'dict' -``` - -But with DeepHash: - -```py ->>> from deepdiff import DeepHash ->>> obj = {1: 2, 'a': 'b'} ->>> DeepHash(obj) -{4355639248: 2468916477072481777512283587789292749, 4355639280: -35787773492556653776377555218122431491, 4358636128: -88390647972316138151822486391929534118, 4358009664: 8833996863197925870419376694314494743, 4357467952: 34150898645750099477987229399128149852} -``` - -So what is exactly the hash of obj in this case? -DeepHash is calculating the hash of the obj and any other object that obj contains. -The output of DeepHash is a dictionary of object IDs to their hashes. -In order to get the hash of obj itself, you need to use the object (or the id of object) to get its hash: - -```py ->>> hashes = DeepHash(obj) ->>> hashes[obj] -34150898645750099477987229399128149852 -``` - -Which you can write as: - -```py ->>> hashes = DeepHash(obj)[obj] -``` - -At first it might seem weird why DeepHash(obj)[obj] but remember that DeepHash(obj) is a dictionary of hashes of all other objects that obj contains too. - - -> - Please take a look at the [DeepHash docs](https://zepworks.com/deepdiff/6.2.3/deephash.html) -> - The full documentation can be found on <https://zepworks.com/deepdiff/6.2.3/> - - -# Using DeepDiff in unit tests - -`result` is the output of the function that is being tests. -`expected` is the expected output of the function. - -```python -self.assertEqual(DeepDiff(expected, result), {}) -``` - -or if you are using Pytest: - - -```python -assert not DeepDiff(expected, result) -``` - -In other words, assert that there is no diff between the expected and the result. - -# Difference with Json Patch - -Unlike [Json Patch](https://tools.ietf.org/html/rfc6902) which is designed only for Json objects, DeepDiff is designed specifically for almost all Python types. In addition to that, DeepDiff checks for type changes and attribute value changes that Json Patch does not cover since there are no such things in Json. Last but not least, DeepDiff gives you the exact path of the item(s) that were changed in Python syntax. - -Example in Json Patch for replacing: - -`{ "op": "replace", "path": "/a/b/c", "value": 42 }` - -Example in DeepDiff for the same operation: - -```python ->>> item1 = {'a':{'b':{'c':'foo'}}} ->>> item2 = {'a':{'b':{'c':42}}} ->>> DeepDiff(item1, item2) -{'type_changes': {"root['a']['b']['c']": {'old_type': <type 'str'>, 'new_value': 42, 'old_value': 'foo', 'new_type': <type 'int'>}}} -``` +If you want to improve the performance of DeepDiff with certain functionalities such as improved json serialization: +`pip install "deepdiff[optimize]"` # Documentation <https://zepworks.com/deepdiff/current/> - -# Pycon 2016 - -I was honored to give a talk about the basics of how DeepDiff does what it does at Pycon 2016. Please check out the video and let me know what you think: - -[Diff It To Dig It Video](https://www.youtube.com/watch?v=J5r99eJIxF4) -And here is more info: <http://zepworks.com/blog/diff-it-to-digg-it/> - # ChangeLog Please take a look at the [CHANGELOG](CHANGELOG.md) file. -# Releases - -We use bump2version to bump and tag releases. - -```bash -git checkout master && git pull -bumpversion {patch|minor|major} -git push && git push --tags -``` - # Contribute 1. Please make your PR against the dev branch @@ -453,11 +73,11 @@ How to cite this library (APA style): - Dehpour, S. (2022). DeepDiff (Version 6.2.3) [Software]. Available from https://github.com/seperman/deepdiff. + Dehpour, S. (2022). DeepDiff (Version 6.3.0) [Software]. Available from https://github.com/seperman/deepdiff. How to cite this library (Chicago style): - Dehpour, Sep. 2022. DeepDiff (version 6.2.3). + Dehpour, Sep. 2022. DeepDiff (version 6.3.0). # Authors diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/deepdiff/__init__.py new/deepdiff-6.3.0/deepdiff/__init__.py --- old/deepdiff-6.2.3/deepdiff/__init__.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/deepdiff/__init__.py 2023-03-17 19:27:19.000000000 +0100 @@ -1,6 +1,6 @@ """This module offers the DeepDiff, DeepSearch, grep, Delta and DeepHash classes.""" # flake8: noqa -__version__ = '6.2.3' +__version__ = '6.3.0' import logging if __name__ == '__main__': diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/deepdiff/commands.py new/deepdiff-6.3.0/deepdiff/commands.py --- old/deepdiff-6.2.3/deepdiff/commands.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/deepdiff/commands.py 2023-03-17 19:27:19.000000000 +0100 @@ -11,6 +11,11 @@ from deepdiff import Delta, DeepSearch, extract as deep_extract from deepdiff.serialization import load_path_content, save_content_to_path +try: + import orjson +except ImportError: + orjson = None + @click.group() def cli(): @@ -105,7 +110,13 @@ # printing into stdout sys.stdout.buffer.write(delta.dumps()) else: - pprint(diff, indent=2) + try: + if orjson: + print(diff.to_json(option=orjson.OPT_INDENT_2)) + else: + print(diff.to_json(indent=2)) + except Exception: + pprint(diff, indent=2) @cli.command() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/deepdiff/diff.py new/deepdiff-6.3.0/deepdiff/diff.py --- old/deepdiff-6.2.3/deepdiff/diff.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/deepdiff/diff.py 2023-03-17 19:27:19.000000000 +0100 @@ -21,7 +21,7 @@ convert_item_or_items_into_compiled_regexes_else_none, type_is_subclass_of_type_group, type_in_type_group, get_doc, number_to_string, datetime_normalize, KEY_TO_VAL_STR, booleans, - np_ndarray, get_numpy_ndarray_rows, OrderedSetPlus, RepeatedTimer, + np_ndarray, np_floating, get_numpy_ndarray_rows, OrderedSetPlus, RepeatedTimer, TEXT_VIEW, TREE_VIEW, DELTA_VIEW, detailed__dict__, add_root_to_paths, np, get_truncate_datetime, dict_, CannotCompare, ENUM_INCLUDE_KEYS) from deepdiff.serialization import SerializationMixin @@ -121,6 +121,8 @@ exclude_obj_callback=None, exclude_obj_callback_strict=None, exclude_paths=None, + include_obj_callback=None, + include_obj_callback_strict=None, include_paths=None, exclude_regex_paths=None, exclude_types=None, @@ -201,6 +203,8 @@ self.ignore_string_case = ignore_string_case self.exclude_obj_callback = exclude_obj_callback self.exclude_obj_callback_strict = exclude_obj_callback_strict + self.include_obj_callback = include_obj_callback + self.include_obj_callback_strict = include_obj_callback_strict self.number_to_string = number_to_string_func or number_to_string self.iterable_compare_func = iterable_compare_func self.ignore_private_variables = ignore_private_variables @@ -464,6 +468,16 @@ (self.exclude_obj_callback_strict(level.t1, level_path) and self.exclude_obj_callback_strict(level.t2, level_path)): skip = True + elif self.include_obj_callback and level_path != 'root': + skip = True + if (self.include_obj_callback(level.t1, level_path) or self.include_obj_callback(level.t2, level_path)): + skip = False + elif self.include_obj_callback_strict and level_path != 'root': + skip = True + if (self.include_obj_callback_strict(level.t1, level_path) and + self.include_obj_callback_strict(level.t2, level_path)): + skip = False + return skip @@ -1308,10 +1322,13 @@ if level.t1 != level.t2: self._report_result('values_changed', level, local_tree=local_tree) - def _diff_numbers(self, level, local_tree=None): + def _diff_numbers(self, level, local_tree=None, report_type_change=True): """Diff Numbers""" - t1_type = "number" if self.ignore_numeric_type_changes else level.t1.__class__.__name__ - t2_type = "number" if self.ignore_numeric_type_changes else level.t2.__class__.__name__ + if report_type_change: + t1_type = "number" if self.ignore_numeric_type_changes else level.t1.__class__.__name__ + t2_type = "number" if self.ignore_numeric_type_changes else level.t2.__class__.__name__ + else: + t1_type = t2_type = '' if self.math_epsilon is not None: if not is_close(level.t1, level.t2, abs_tol=self.math_epsilon): @@ -1489,8 +1506,8 @@ if self._skip_this(level): return + report_type_change = True if get_type(level.t1) != get_type(level.t2): - report_type_change = True for type_group in self.ignore_type_in_groups: if self.type_check_func(level.t1, type_group) and self.type_check_func(level.t2, type_group): report_type_change = False @@ -1503,7 +1520,7 @@ self._report_result('values_changed', level, local_tree=local_tree) return - if self.ignore_nan_inequality and isinstance(level.t1, float) and str(level.t1) == str(level.t2) == 'nan': + if self.ignore_nan_inequality and isinstance(level.t1, (float, np_floating)) and str(level.t1) == str(level.t2) == 'nan': return if isinstance(level.t1, booleans): @@ -1519,7 +1536,7 @@ self._diff_uuids(level, local_tree=local_tree) elif isinstance(level.t1, numbers): - self._diff_numbers(level, local_tree=local_tree) + self._diff_numbers(level, local_tree=local_tree, report_type_change=report_type_change) elif isinstance(level.t1, Mapping): self._diff_dict(level, parents_ids, local_tree=local_tree) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/deepdiff/helper.py new/deepdiff-6.3.0/deepdiff/helper.py --- old/deepdiff-6.2.3/deepdiff/helper.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/deepdiff/helper.py 2023-03-17 19:27:19.000000000 +0100 @@ -39,6 +39,7 @@ np_float32 = np_type # pragma: no cover. np_float64 = np_type # pragma: no cover. np_float_ = np_type # pragma: no cover. + np_floating = np_type # pragma: no cover. np_complex64 = np_type # pragma: no cover. np_complex128 = np_type # pragma: no cover. np_complex_ = np_type # pragma: no cover. @@ -60,6 +61,7 @@ np_float32 = np.float32 np_float64 = np.float64 np_float_ = np.float_ + np_floating = np.floating np_complex64 = np.complex64 np_complex128 = np.complex128 np_complex_ = np.complex_ @@ -68,7 +70,7 @@ numpy_numbers = ( np_int8, np_int16, np_int32, np_int64, np_uint8, np_uint16, np_uint32, np_uint64, np_intp, np_uintp, - np_float32, np_float64, np_float_, np_complex64, + np_float32, np_float64, np_float_, np_floating, np_complex64, np_complex128, np_complex_,) numpy_complex_numbers = ( @@ -336,7 +338,7 @@ using = number_formatting[number_format_notation] except KeyError: raise ValueError("number_format_notation got invalid value of {}. The valid values are 'f' and 'e'".format(number_format_notation)) from None - + if not isinstance(number, numbers): return number elif isinstance(number, Decimal): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/deepdiff/model.py new/deepdiff-6.3.0/deepdiff/model.py --- old/deepdiff-6.2.3/deepdiff/model.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/deepdiff/model.py 2023-03-17 19:27:19.000000000 +0100 @@ -86,6 +86,7 @@ self['iterable_item_added'].remove(level_after) level_before.t2 = level_after.t2 self['values_changed'].add(level_before) + level_before.report_type = 'values_changed' if 'iterable_item_removed' in self and not self['iterable_item_removed']: del self['iterable_item_removed'] if 'iterable_item_added' in self and not self['iterable_item_added']: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/deepdiff/operator.py new/deepdiff-6.3.0/deepdiff/operator.py --- old/deepdiff-6.2.3/deepdiff/operator.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/deepdiff/operator.py 2023-03-17 19:27:19.000000000 +0100 @@ -25,3 +25,14 @@ def give_up_diffing(self, level, diff_instance) -> bool: raise NotImplementedError('Please implement the diff function.') + + +class PrefixOrSuffixOperator: + + def match(self, level) -> bool: + return level.t1 and level.t2 and isinstance(level.t1, str) and isinstance(level.t2, str) + + def give_up_diffing(self, level, diff_instance) -> bool: + t1 = level.t1 + t2 = level.t2 + return t1.startswith(t2) or t2.startswith(t1) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/deepdiff/serialization.py new/deepdiff-6.3.0/deepdiff/serialization.py --- old/deepdiff-6.2.3/deepdiff/serialization.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/deepdiff/serialization.py 2023-03-17 19:27:19.000000000 +0100 @@ -3,7 +3,6 @@ import io import os import json -import orjson import uuid import logging import re # NOQA @@ -26,6 +25,15 @@ except ImportError: # pragma: no cover. import csv clevercsv = None # pragma: no cover. +try: + import orjson +except ImportError: # pragma: no cover. + orjson = None +try: + from pydantic import BaseModel as PydanticBaseModel +except ImportError: # pragma: no cover. + PydanticBaseModel = None + from copy import deepcopy from functools import partial from collections.abc import Mapping @@ -303,7 +311,7 @@ return None -def pickle_dump(obj, file_obj=None): +def pickle_dump(obj, file_obj=None, protocol=4): """ **pickle_dump** Dumps the obj into pickled content. @@ -321,21 +329,21 @@ """ file_obj_passed = bool(file_obj) file_obj = file_obj or io.BytesIO() - # We expect at least python 3.5 so protocol 4 is good. - _RestrictedPickler(file_obj, protocol=4, fix_imports=False).dump(obj) + _RestrictedPickler(file_obj, protocol=protocol, fix_imports=False).dump(obj) if not file_obj_passed: return file_obj.getvalue() -def pickle_load(content, safe_to_import=None): +def pickle_load(content=None, file_obj=None, safe_to_import=None): """ **pickle_load** Load the pickled content. content should be a bytes object. **Parameters** - content : Bytes of pickled object. It needs to have Delta header in it that is - separated by a newline character from the rest of the pickled object. + content : Bytes of pickled object. + + file_obj : A file object to load the content from safe_to_import : A set of modules that needs to be explicitly allowed to be loaded. Example: {'mymodule.MyClass', 'decimal.Decimal'} @@ -354,9 +362,13 @@ """ + if not content and not file_obj: + raise ValueError('Please either pass the content or the file_obj to pickle_load.') if isinstance(content, str): content = content.encode('utf-8') - return _RestrictedUnpickler(io.BytesIO(content), safe_to_import=safe_to_import).load() + if content: + file_obj = io.BytesIO(content) + return _RestrictedUnpickler(file_obj, safe_to_import=safe_to_import).load() def _get_pretty_form_text(verbose_level): @@ -522,6 +534,9 @@ uuid.UUID: lambda x: str(x), } +if PydanticBaseModel: + JSON_CONVERTOR[PydanticBaseModel] = lambda x: x.dict() + def json_convertor_default(default_mapping=None): if default_mapping: @@ -556,15 +571,20 @@ def json_dumps(item, default_mapping=None, **kwargs): """ Dump json with extra details that are not normally json serializable - - Note: I tried to replace json with orjson for its speed. It does work - but the output it makes is a byte object and Postgres couldn't directly use it without - encoding to str. So I switched back to json. - """ - return orjson.dumps( - item, - default=json_convertor_default(default_mapping=default_mapping), - **kwargs).decode(encoding='utf-8') + """ + if orjson: + indent = kwargs.pop('indent', None) + if indent: + kwargs['option'] = orjson.OPT_INDENT_2 + return orjson.dumps( + item, + default=json_convertor_default(default_mapping=default_mapping), + **kwargs).decode(encoding='utf-8') + else: + return json.dumps( + item, + default=json_convertor_default(default_mapping=default_mapping), + **kwargs) json_loads = partial(json.loads, cls=JSONDecoder) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/docs/authors.rst new/deepdiff-6.3.0/docs/authors.rst --- old/deepdiff-6.2.3/docs/authors.rst 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/docs/authors.rst 2023-03-17 19:27:19.000000000 +0100 @@ -66,6 +66,12 @@ np.array_equal - `martin-kokos <https://github.com/martin-kokos>`__ for using Pytestâs tmp_path fixture instead of /tmp/ +- HÃ¥vard Thom `havardthom <https://github.com/havardthom>`__ for adding + include_obj_callback and include_obj_callback_strict. +- `Noam Gottlieb <https://github.com/noamgot>`__ for fixing a corner + case where numpyâs ``np.float32`` nans are not ignored when using + ``ignore_nan_equality``. + .. _Sep Dehpour (Seperman): http://www.zepworks.com .. _Victor Hahn Castell: http://hahncastell.de diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/docs/changelog.rst new/deepdiff-6.3.0/docs/changelog.rst --- old/deepdiff-6.2.3/docs/changelog.rst 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/docs/changelog.rst 2023-03-17 19:27:19.000000000 +0100 @@ -5,6 +5,19 @@ DeepDiff Changelog +- v6-3-0 + + - ``PrefixOrSuffixOperator``: This operator will skip strings that + are suffix or prefix of each other. + - ``include_obj_callback`` and ``include_obj_callback_strict`` are + added by `HÃ¥vard Thom <https://github.com/havardthom>`__. + - Fixed a corner case where numpyâs ``np.float32`` nans are not + ignored when using ``ignore_nan_equality`` by `Noam + Gottlieb <https://github.com/noamgot>`__ + - ``orjson`` becomes optional again. + - Fix for ``ignore_type_in_groups`` with numeric values so it does + not report number changes when the number types are different. + - v6-2-3 - Switching to Orjson for serialization to improve the performance. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/docs/conf.py new/deepdiff-6.3.0/docs/conf.py --- old/deepdiff-6.2.3/docs/conf.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/docs/conf.py 2023-03-17 19:27:19.000000000 +0100 @@ -60,9 +60,9 @@ # built documents. # # The short X.Y version. -version = '6.2.3' +version = '6.3.0' # The full version, including alpha/beta/rc tags. -release = '6.2.3' +release = '6.3.0' load_dotenv(override=True) DOC_VERSION = os.environ.get('DOC_VERSION', version) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/docs/custom.rst new/deepdiff-6.3.0/docs/custom.rst --- old/deepdiff-6.2.3/docs/custom.rst 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/docs/custom.rst 2023-03-17 19:27:19.000000000 +0100 @@ -128,21 +128,62 @@ Custom Operators ---------------- -Whether two objects are different or not are largely depend on the context. For example, apple and banana are the same +Whether two objects are different or not largely depends on the context. For example, apples and bananas are the same if you are considering whether they are fruits or not. In that case, you can pass a *custom_operators* for the job. -In fact, custom operators give you a lot of power. In the following examples we explore use cases from making DeepDiff -report the L2 Distance of items, to only include certain paths in diffing all the way to making DeepDiff stop diffing -as soon as the first diff is reported. +Custom operators give you a lot of power. In the following examples, we explore various use cases such as: + +- Making DeepDiff report the L2 Distance of items +- Only include specific paths in diffing +- Making DeepDiff stop diffing once we find the first diff. + +You can use one of the predefined custom operators that come with DeepDiff. Or you can define one yourself. + + +Built-In Custom Operators + +.. _prefix_or_suffix_operator_label: + +PrefixOrSuffixOperator +...................... + + +This operator will skip strings that are suffix or prefix of each other. + +For example when this operator is used, the two strings of "joe" and "joe's car" will not be reported as different. + + >>> from deepdiff import DeepDiff + >>> from deepdiff.operator import PrefixOrSuffixOperator + >>> t1 = { + ... "key1": ["foo", "bar's food", "jack", "joe"] + ... } + >>> t2 = { + ... "key1": ["foo", "bar", "jill", "joe'car"] + ... } + >>> + >>> DeepDiff(t1, t2) + {'values_changed': {"root['key1'][1]": {'new_value': 'bar', 'old_value': "bar's food"}, "root['key1'][2]": {'new_value': 'jill', 'old_value': 'jack'}, "root['key1'][3]": {'new_value': "joe'car", 'old_value': 'joe'}}} + >>> DeepDiff(t1, t2, custom_operators=[ + ... PrefixOrSuffixOperator() + ... ]) + >>> + {'values_changed': {"root['key1'][2]": {'new_value': 'jill', 'old_value': 'jack'}}} + + + + +Define A Custom Operator +------------------------ + To define an custom operator, you just need to inherit a *BaseOperator* and * implement a give_up_diffing method * give_up_diffing(level: DiffLevel, diff_instance: DeepDiff) -> boolean - If it returns True, then we will give up diffing the 2 objects. + If it returns True, then we will give up diffing the tow objects. You may or may not use the diff_instance.custom_report_result within this function to report any diff. If you decide not to report anything, and this function returns True, then the objects are basically skipped in the results. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/docs/delta.rst new/deepdiff-6.3.0/docs/delta.rst --- old/deepdiff-6.2.3/docs/delta.rst 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/docs/delta.rst 2023-03-17 19:27:19.000000000 +0100 @@ -153,16 +153,16 @@ If all you deal with are Json serializable objects, you can use json for serialization. >>> from deepdiff import DeepDiff, Delta ->>> import json +>>> from deepdiff.serialization import json_dumps, json_loads >>> t1 = {"a": 1} >>> t2 = {"a": 2} >>> >>> diff = DeepDiff(t1, t2) ->>> delta = Delta(diff, serializer=json.dumps) +>>> delta = Delta(diff, serializer=json_dumps) >>> dump = delta.dumps() >>> dump '{"values_changed": {"root[\'a\']": {"new_value": 2}}}' ->>> delta_reloaded = Delta(dump, deserializer=json.loads) +>>> delta_reloaded = Delta(dump, deserializer=json_loads) >>> t2 == delta_reloaded + t1 True diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/docs/diff_doc.rst new/deepdiff-6.3.0/docs/diff_doc.rst --- old/deepdiff-6.2.3/docs/diff_doc.rst 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/docs/diff_doc.rst 2023-03-17 19:27:19.000000000 +0100 @@ -62,7 +62,16 @@ exclude_obj_callback_strict: function, default = None :ref:`exclude_obj_callback_strict_label` - A function that works the same way as exclude_obj_callback, but excludes elements from the result only if the function returns True for both elements + A function that works the same way as exclude_obj_callback, but excludes elements from the result only if the function returns True for both elements. + +include_obj_callback: function, default = None + :ref:`include_obj_callback_label` + A function that takes the object and its path and returns a Boolean. If True is returned, the object is included in the results, otherwise it is excluded. + This is to give the user a higher level of control than one can achieve via include_paths. + +include_obj_callback_strict: function, default = None + :ref:`include_obj_callback_strict_label` + A function that works the same way as include_obj_callback, but includes elements in the result only if the function returns True for both elements. get_deep_distance: Boolean, default = False :ref:`get_deep_distance_label` will get you the deep distance between objects. The distance is a number between 0 and 1 where zero means there is no diff between the 2 objects and 1 means they are very different. Note that this number should only be used to compare the similarity of 2 objects and nothing more. The algorithm for calculating this number may or may not change in the future releases of DeepDiff. diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/docs/ignore_types_or_values.rst new/deepdiff-6.3.0/docs/ignore_types_or_values.rst --- old/deepdiff-6.2.3/docs/ignore_types_or_values.rst 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/docs/ignore_types_or_values.rst 2023-03-17 19:27:19.000000000 +0100 @@ -291,6 +291,44 @@ >>> DeepDiff(t1, t2, exclude_obj_callback_strict=exclude_obj_callback_strict) {'values_changed': {"root['x']": {'new_value': 12, 'old_value': 10}}} + +.. _include_obj_callback_label: + +Include Obj Callback +-------------------- + +include_obj_callback: function, default = None + A function that takes the object and its path and returns a Boolean. If True is returned, the object is included in the results, otherwise it is excluded. + This is to give the user a higher level of control than one can achieve via include_paths. + + >>> def include_obj_callback(obj, path): + ... return True if "include" in path or isinstance(obj, int) else False + ... + >>> t1 = {"x": 10, "y": "b", "z": "c", "include_me": "a"} + >>> t2 = {"x": 10, "y": "b", "z": "c", "include_me": "b"} + >>> DeepDiff(t1, t2, include_obj_callback=include_obj_callback) + {'values_changed': {"root['include_me']": {'new_value': "b", 'old_value': "a"}}} + + +.. _include_obj_callback_strict_label: + +Include Obj Callback Strict +--------------------------- + +include_obj_callback_strict: function, default = None + A function that works the same way as include_obj_callback, but includes elements in the result only if the function returns True for both elements. + + >>> def include_obj_callback_strict(obj, path): + ... return True if isinstance(obj, int) and obj > 10 else False + ... + >>> t1 = {"x": 10, "y": "b", "z": "c"} + >>> t1 = {"x": 12, "y": "b", "z": "c"} + >>> DeepDiff(t1, t2, include_obj_callback=include_obj_callback_strict) + {'values_changed': {"root['x']": {'new_value': 12, 'old_value': 10}}} + >>> DeepDiff(t1, t2, include_obj_callback_strict=include_obj_callback_strict) + {} + + .. _truncate_datetime_label: Truncate Datetime diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/docs/index.rst new/deepdiff-6.3.0/docs/index.rst --- old/deepdiff-6.2.3/docs/index.rst 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/docs/index.rst 2023-03-17 19:27:19.000000000 +0100 @@ -4,12 +4,12 @@ contain the root `toctree` directive. -DeepDiff 6.2.3 documentation! +DeepDiff 6.3.0 documentation! ============================= -***************** -DeepDiff Overview -***************** +******* +Modules +******* The DeepDiff library includes the following modules: @@ -31,32 +31,25 @@ What is New *********** -DeepDiff 6-2-0 --------------- - -- Major improvement in the diff report for lists when items are all hashable and the order of items is important. -DeepDiff 6-1-0 +DeepDiff 6-3-0 -------------- -- DeepDiff.affected_paths can be used to get the list of all paths - where a change, addition, or deletion was reported for. -- DeepDiff.affected_root_keys can be used to get the list of all paths - where a change, addition, or deletion was reported for. -- Bugfix: ValueError when using Decimal 0.x #339 by `Enric - Pou <https://github.com/epou>`__ -- Serialization of UUID +- :ref:`prefix_or_suffix_operator_label`: This operator will skip strings that are + suffix or prefix of each other. +- :ref:`include_obj_callback_label` and :ref:`include_obj_callback_strict_label` are + added by `HÃ¥vard Thom <https://github.com/havardthom>`__. +- Fixed a corner case where numpyâs ``np.float32`` nans are not ignored + when using ``ignore_nan_equality`` by `Noam + Gottlieb <https://github.com/noamgot>`__ +- ``orjson`` becomes optional again. +- Fix for ``ignore_type_in_groups`` with numeric values so it does not report number changes when the number types are different. -DeepDiff 6-0-0 +DeepDiff 6-2-0 -------------- -- :ref:`exclude_obj_callback_strict_label` - parameter is added to DeepDiff by Mikhail Khviyuzov - `mskhviyu <https://github.com/mskhviyu>`__. -- A fix for diffing using ``iterable_compare_func`` with nested objects - by `dtorres-sf <https://github.com/dtorres-sf>`__ who originally - contributed this feature. -Note: There are no breaking changes in DeepDiff 6 compared to the latest DeepDiff 5 releases. +- Major improvement in the diff report for lists when items are all hashable and the order of items is important. + ********* Tutorials @@ -76,6 +69,10 @@ pip install "deepdiff[cli]" +If you want to improve the performance of DeepDiff with certain processes such as json serialization:: + + pip install "deepdiff[optimize]" + Read about DeepDiff optimizations at :ref:`optimizations_label` Importing @@ -118,23 +115,13 @@ $ deep extract --help + Supported data types ~~~~~~~~~~~~~~~~~~~~ int, string, unicode, dictionary, list, tuple, set, frozenset, OrderedDict, NamedTuple, Numpy, custom objects and more! -***** -Pycon -***** - -**Pycon 2016 Talk** -A talk was given about the basics of how DeepDiff does what it does at Pycon 2016. -`Diff it to Dig it Pycon 2016 video <https://www.youtube.com/watch?v=J5r99eJIxF4>`_ - -You can find more information about the contents of that Pycon talk here: http://zepworks.com/blog/diff-it-to-digg-it/ - - References ========== diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/docs/optimizations.rst new/deepdiff-6.3.0/docs/optimizations.rst --- old/deepdiff-6.2.3/docs/optimizations.rst 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/docs/optimizations.rst 2023-03-17 19:27:19.000000000 +0100 @@ -8,6 +8,15 @@ If you are dealing with large nested objects and ignore_order=True, chances are DeepDiff takes a while to calculate the diff. Here are some tips that may help you with optimizations and progress report. +Optimized Libraries +------------------- + +If you dump DeepDiff or Delta objects as json, you can improve the performance by installing orjson. +DeepDiff will automatically use orjson instead of Python's built-in json library to do json serialization. + + pip install "deepdiff[optimize]" + + Max Passes ---------- diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/requirements-dev-3.7.txt new/deepdiff-6.3.0/requirements-dev-3.7.txt --- old/deepdiff-6.2.3/requirements-dev-3.7.txt 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/requirements-dev-3.7.txt 2023-03-17 19:27:19.000000000 +0100 @@ -1,4 +1,3 @@ -wheel==0.37.0 -r requirements.txt -r requirements-cli.txt bump2version==1.0.1 @@ -8,3 +7,4 @@ pytest==7.1.2 python-dotenv==0.20.0 python-dateutil==2.8.2 +wheel==0.38.1 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/requirements-dev.txt new/deepdiff-6.3.0/requirements-dev.txt --- old/deepdiff-6.2.3/requirements-dev.txt 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/requirements-dev.txt 2023-03-17 19:27:19.000000000 +0100 @@ -1,4 +1,3 @@ -wheel==0.38.4 -r requirements.txt -r requirements-cli.txt bump2version==1.0.1 @@ -14,3 +13,5 @@ sphinx-sitemap==2.2.1 flake8==6.0.0 python-dateutil==2.8.2 +orjson==3.8.3 +wheel==0.38.1 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/requirements-optimize.txt new/deepdiff-6.3.0/requirements-optimize.txt --- old/deepdiff-6.2.3/requirements-optimize.txt 1970-01-01 01:00:00.000000000 +0100 +++ new/deepdiff-6.3.0/requirements-optimize.txt 2023-03-17 19:27:19.000000000 +0100 @@ -0,0 +1 @@ +orjson diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/requirements.txt new/deepdiff-6.3.0/requirements.txt --- old/deepdiff-6.2.3/requirements.txt 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/requirements.txt 2023-03-17 19:27:19.000000000 +0100 @@ -1,2 +1 @@ ordered-set>=4.0.2,<4.2.0 -orjson diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/setup.cfg new/deepdiff-6.3.0/setup.cfg --- old/deepdiff-6.2.3/setup.cfg 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/setup.cfg 2023-03-17 19:27:19.000000000 +0100 @@ -1,5 +1,5 @@ [bumpversion] -current_version = 6.2.3 +current_version = 6.3.0 commit = True tag = True tag_name = {new_version} diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/setup.py new/deepdiff-6.3.0/setup.py --- old/deepdiff-6.2.3/setup.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/setup.py 2023-03-17 19:27:19.000000000 +0100 @@ -10,7 +10,7 @@ if os.environ.get('USER', '') == 'vagrant': del os.link -version = '6.2.3' +version = '6.3.0' def get_reqs(filename): @@ -21,6 +21,7 @@ reqs = get_reqs("requirements.txt") cli_reqs = get_reqs("requirements-cli.txt") +optimize_reqs = get_reqs("requirements-optimize.txt") with open('README.md') as file: long_description = file.read() @@ -45,6 +46,7 @@ python_requires='>=3.7', extras_require={ "cli": cli_reqs, + "optimize": optimize_reqs, }, classifiers=[ "Intended Audience :: Developers", @@ -54,6 +56,7 @@ "Programming Language :: Python :: 3.8", "Programming Language :: Python :: 3.9", "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", "Programming Language :: Python :: Implementation :: PyPy", "Development Status :: 5 - Production/Stable", "License :: OSI Approved :: MIT License" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/tests/test_command.py new/deepdiff-6.3.0/tests/test_command.py --- old/deepdiff-6.2.3/tests/test_command.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/tests/test_command.py 2023-03-17 19:27:19.000000000 +0100 @@ -11,14 +11,14 @@ class TestCommands: @pytest.mark.parametrize('t1, t2, expected_in_stdout, expected_exit_code', [ - ('t1.json', 't2.json', "'dictionary_item_added\': [root[0]", 0), + ('t1.json', 't2.json', '"dictionary_item_added": [\n "root[0]', 0), ('t1_corrupt.json', 't2.json', "Expecting property name enclosed in double quotes", 1), - ('t1.json', 't2_json.csv', "'old_value\': \'value2\'", 0), - ('t2_json.csv', 't1.json', "'old_value\': \'value3\'", 0), - ('t1.csv', 't2.csv', "\'new_value\': \'James\'", 0), + ('t1.json', 't2_json.csv', '"old_value": "value2"', 0), + ('t2_json.csv', 't1.json', '"old_value": "value3"', 0), + ('t1.csv', 't2.csv', '"new_value": "James"', 0), ('t1.toml', 't2.toml', "10.0.0.2", 0), - ('t1.pickle', 't2.pickle', "'new_value': 5, 'old_value': 1", 0), - ('t1.yaml', 't2.yaml', "'new_value': 61, 'old_value': 65", 0), + ('t1.pickle', 't2.pickle', '"new_value": 5,\n "old_value": 1', 0), + ('t1.yaml', 't2.yaml', '"new_value": 61,\n "old_value": 65', 0), ]) def test_diff_command(self, t1, t2, expected_in_stdout, expected_exit_code): t1 = os.path.join(FIXTURES_DIR, t1) @@ -74,7 +74,7 @@ diffed = runner.invoke(diff, [t1, t2, '--group-by', 'id']) assert 0 == diffed.exit_code assert 'values_changed' in diffed.output - assert '\'new_value\': \'Chicken\'' in diffed.output + assert '"new_value": "Chicken"' in diffed.output def test_command_math_epsilon(self): t1 = os.path.join(FIXTURES_DIR, 'd_t1.yaml') @@ -86,7 +86,7 @@ diffed2 = runner.invoke(diff, [t1, t2, '--math-epsilon', '0.001']) assert 0 == diffed2.exit_code - assert "{'values_changed': {'root[2][2]': {'new_value': 0.289, 'old_value': 0.288}}}\n" == diffed2.output + assert '{\n "values_changed": {\n "root[2][2]": {\n "new_value": 0.289,\n "old_value": 0.288\n }\n }\n}\n' == diffed2.output def test_command_grep(self): path = os.path.join(FIXTURES_DIR, 'd_t1.yaml') diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/tests/test_delta.py new/deepdiff-6.3.0/tests/test_delta.py --- old/deepdiff-6.2.3/tests/test_delta.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/tests/test_delta.py 2023-03-17 19:27:19.000000000 +0100 @@ -1188,6 +1188,8 @@ 'exclude_types_tuple': None, 'ignore_type_subclasses': False, 'ignore_string_case': False, + 'include_obj_callback': None, + 'include_obj_callback_strict': None, 'exclude_obj_callback': None, 'exclude_obj_callback_strict': None, 'ignore_private_variables': True, diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/tests/test_diff_numpy.py new/deepdiff-6.3.0/tests/test_diff_numpy.py --- old/deepdiff-6.2.3/tests/test_diff_numpy.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/tests/test_diff_numpy.py 2023-03-17 19:27:19.000000000 +0100 @@ -105,6 +105,14 @@ } }, }, + 'numpy_array9_ignore_nan_inequality_float32': { + 't1': np.array([1, 2, 3, np.nan], np.float32), + 't2': np.array([1, 2, 4, np.nan], np.float32), + 'deepdiff_kwargs': { + 'ignore_nan_inequality': True, + }, + 'expected_result': {'values_changed': {'root[2]': {'new_value': 4.0, 'old_value': 3.0}}} + }, 'numpy_almost_equal': { 't1': np.array([1.0, 2.3333333333333]), 't2': np.array([1.0, 2.33333334]), diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/tests/test_diff_text.py new/deepdiff-6.3.0/tests/test_diff_text.py --- old/deepdiff-6.2.3/tests/test_diff_text.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/tests/test_diff_text.py 2023-03-17 19:27:19.000000000 +0100 @@ -1196,6 +1196,10 @@ result = {'values_changed': {'root[4]': {'new_value': 'now', 'old_value': now}}} assert result == ddiff + def test_ignore_type_in_groups_float_vs_decimal(self): + diff = DeepDiff(float('0.1'), Decimal('0.1'), ignore_type_in_groups=[(float, Decimal)], significant_digits=2) + assert not diff + @pytest.mark.parametrize("t1, t2, significant_digits, result", [ ([0.1], [Decimal('0.10')], 55, {'values_changed': {'root[0]': {'new_value': Decimal('0.10'), 'old_value': 0.1}}}), # Due to floating point arithmetics with high significant digits. @@ -1396,6 +1400,30 @@ result = {} assert result == ddiff + def test_include_obj_callback(self): + def include_obj_callback(obj, path): + return True if "include" in path or isinstance(obj, int) else False + + t1 = {"x": 10, "y": "b", "z": "c", "include_me": "a"} + t2 = {"x": 10, "y": "c", "z": "b", "include_me": "b"} + ddiff = DeepDiff(t1, t2, include_obj_callback=include_obj_callback) + result = {'values_changed': {"root['include_me']": {'new_value': "b", 'old_value': "a"}}} + assert result == ddiff + assert {"root['include_me']"} == ddiff.affected_paths + assert {"include_me"} == ddiff.affected_root_keys + + def test_include_obj_callback_strict(self): + def include_obj_callback_strict(obj, path): + return True if isinstance(obj, int) and obj > 10 else False + + t1 = {"x": 11, "y": 10, "z": "c"} + t2 = {"x": 12, "y": 12, "z": "c"} + ddiff = DeepDiff(t1, t2, include_obj_callback_strict=include_obj_callback_strict) + result = {'values_changed': {"root['x']": {'new_value': 12, 'old_value': 11}}} + assert result == ddiff + assert {"root['x']"} == ddiff.affected_paths + assert {"x"} == ddiff.affected_root_keys + def test_skip_exclude_obj_callback(self): def exclude_obj_callback(obj, path): return True if "skip" in path or isinstance(obj, int) else False diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/tests/test_diff_tree.py new/deepdiff-6.3.0/tests/test_diff_tree.py --- old/deepdiff-6.2.3/tests/test_diff_tree.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/tests/test_diff_tree.py 2023-03-17 19:27:19.000000000 +0100 @@ -121,6 +121,14 @@ assert change.path(force='yes') == 'root(unrepresentable)' assert change.path(force='fake') == 'root[2]' + def test_report_type_in_iterable(self): + a = {"temp": ["a"]} + b = {"temp": ["b"]} + + ddiff = DeepDiff(a, b, ignore_order=True, view="tree") + report_type = ddiff['values_changed'][0].report_type + assert 'values_changed' == report_type + def test_significant_digits(self): ddiff = DeepDiff( [0.012, 0.98], diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/deepdiff-6.2.3/tests/test_operators.py new/deepdiff-6.3.0/tests/test_operators.py --- old/deepdiff-6.2.3/tests/test_operators.py 2023-01-06 06:00:31.000000000 +0100 +++ new/deepdiff-6.3.0/tests/test_operators.py 2023-03-17 19:27:19.000000000 +0100 @@ -2,7 +2,7 @@ from typing import List from deepdiff import DeepDiff -from deepdiff.operator import BaseOperator +from deepdiff.operator import BaseOperator, PrefixOrSuffixOperator class TestOperators: @@ -217,3 +217,26 @@ expected = {'values_changed': {'root[0][1]': {'new_value': 3, 'old_value': 2}}} assert expected == ddiff + + def test_prefix_or_suffix_diff(self): + + t1 = { + "key1": ["foo", "bar's food", "jack", "joe"] + } + t2 = { + "key1": ["foo", "bar", "jill", "joe'car"] + } + + ddiff = DeepDiff(t1, t2, custom_operators=[ + PrefixOrSuffixOperator() + ]) + + expected = {'values_changed': {"root['key1'][2]": {'new_value': 'jill', 'old_value': 'jack'}}} + assert expected == ddiff + + ddiff2 = DeepDiff(t1, t2, ignore_order=True, custom_operators=[ + PrefixOrSuffixOperator() + ]) + + expected2 = {'values_changed': {"root['key1'][2]": {'new_value': 'jill', 'old_value': 'jack'}}} + assert expected2 == ddiff2