Paul,
I agree that the mapping to a flat data structure might have limited use
cases. However, like you said it is only one example. What I am proposing
is a nested iteration tool that can map both keys and values to new keys
and new values. It only considers nesting in dictionaries, lists, and
tuples by default, but allows users to pass another function to continue
searching for pointers in other types. For example:
class Node:
name: str=""
next: 'Node' = None
val: Any = None
def newDefaultMapper(o) -> o:
'''Converts the object o and returns it'''
if isinstance(o, dict): # This would be automatically implemented in
itermapper.Mapper if arg 'dictMapper' was defined
return {k+"Mapped": itermapper.Mapper(v) for k, v in o.items()}
if isinstance(o, Node):
o.name = 'm' + o.name[1]
if o.next:
return itermapper.Mapper(o.next) # resorts to newDefaultMapper
if it cannot handle, then the oldDefaultMapper
return o
return itermapper.oldDefaultMapper(o)
n1 = Node(name='n1')
n2 = Node(name='n2')
n1.next = n2
d = {'a': 1, 'b': n1}
itermapper(d, default=newDefaultMapper)
print(d)
>> {'aMapped': 1, 'bMapped': <__main__.Node at 0x....>}
print(n1.name, n2.name)
>> m1 m2
This functionality partially exists in the current versions of python in
the JSON dumps function. Dumps iterates over all values in a nested data
structure and it also accepts a JSONEncoder
<https://docs.python.org/3/library/json.html#json.JSONEncoder> with a new
default function to handle arbitrary types. No such functionality exists in
other parts of python to map values in a nested data structure into values
of another identical nested data structure.
What I am proposing goes beyond this though to also handle keys. It also
doesn't return JSON strings, but a new mapped data structure.
Best,
Sven
On Wed, Jul 21, 2021 at 9:30 AM Paul Moore <p.f.mo...@gmail.com> wrote:
> You've given some examples of what flatten() does, but I still don't
> really understand how it's expected to work. Could you provide a
> specification, rather than just examples? Also, I've never needed a
> function like this in anything I've ever done - that may mean nothing,
> but it does suggest that you need to demonstrate that it's a generally
> useful tool (maybe by pointing to examples of "real world" code in
> various contexts that would be improved by using it). There's a lot of
> pressure to keep the stdlib for things that are widely applicable, and
> the burden of demonstrating that lies with the proposer.
>
> As the proposal stands here, I honestly don't see anything that
> suggests this wouldn't be better as a standalone function published on
> PyPI.
>
> Paul
>
> On Wed, 21 Jul 2021 at 13:43, Sven Voigt <svenpvo...@gmail.com> wrote:
> >
> > Hello Python-ideas,
> >
> > I am working with lots of JSON objects in my work and need to obtain
> JSON pointers to particular components of a JSON object. These pointers are
> defined as part of the JSON-LD specifications as relative IRIs and as part
> of the JSON-Schema specifications. However, I am not aware of these
> pointers being part of the JSON specification. The pointers are really
> straightforward and follow the same syntax as other IRIs (URLs) or file
> paths. For example, the following flatten function illustrates the mapping
> of nested dictionaries and arrays to a single flat dictionary, where each
> nested key is mapped to a unique path.
> >
> > d = {"a": 1, "b": {"ba": 2, "bb": [{"bba": 3, "bbb": None}]}}
> > flatten(d)
> > >> {'a': 1, 'b/ba': 2, 'b/bb/0/bba': 3, 'b/bb/0/bbb': None}
> >
> > Not only does this conversion help in generating JSON pointers, but it
> helps with logic for nested data structures. Specifically, assume there is
> a dictionary, which includes nested dictionaries, lists, and tuples, and
> any of these elements are contained within another dictionary or that other
> dictionaries elements. Then it is not sufficient to simply compare the
> items of the two dictionaries to check whether the first is a subset of the
> other. However, the flat structures can be compared directly.
> >
> > 1. A nested dictionary is a subset of another nested dictionary. The
> flatten function ensures the nested dictionaries are not checked for
> equivalence but subset of.
> > # Current
> > {"a": 1, "c": {"d": 4}}.items() <= {"a": 1, "b": 2, "c": {"d": 4, "e":
> 5}}.items()
> > >> False
> > # Proposed
> > flatten({"a": 1, "c": {"d": 4}}).items() <= flatten({"a": 1, "b": 2,
> "c": {"d": 4, "e": 5}}).items()
> > >> True
> >
> > 2. A nested list or tuple is a subset of a dictionary.
> > # Current
> > {"a": 1, "c": [3]}.items() <= {"a": 1, "b": 2, "c": [3,3]}.items()
> > >> False
> > # Proposed
> > flatten({"a": 1, "c": [3]}).items() <= flatten({"a": 1, "b": 2, "c":
> [3,3]}).items()
> > >> True
> >
> > Note that these examples only have one level of nesting, but the flatten
> function must handle any level of nesting. For the current version of the
> flatten function, I essentially borrowed the JSON "dumps" source code.
> However, the dumps code doesn't utilize any other functions and I believe
> there may be a missing "nestedIter" function in python. This function could
> be used for JSON dumps or flatten or anytime someone wants to visit all
> nested elements of a dictionary and perform some operation. Therefore, I
> think this function could be included in the JSON library if python-ideas
> thinks that the pointers are very specific to JSON or the itertools
> library, where the nested iteration function could be generally used for
> all nested data types.
> >
> > Let me know what you think of this proposal and I am looking forward to
> your responses.
> >
> > Best,
> > Sven
> > _______________________________________________
> > Python-ideas mailing list -- python-ideas@python.org
> > To unsubscribe send an email to python-ideas-le...@python.org
> > https://mail.python.org/mailman3/lists/python-ideas.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/V2DNTLXII2QNYSE3FJXRO45ZFTCMBZ5G/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/OSGNAUNDDD64P4BIN6BNW5JO4HBJN4V5/
Code of Conduct: http://python.org/psf/codeofconduct/
