> > here's the crash now:
> >
> > Traceback (most recent call last):
> > File
> > "/nix/store/f2krmq3iv5nibcvn4rw7nrnrciqprdkh-python3-3.12.9/lib/python3.12/pdb.py",
> > line 1960, in main
> > pdb._run(target)
> > File
> > "/nix/store/f2krmq3iv5nibcvn4rw7nrnrciqprdkh-python3-3.12.9/lib/python3.12/pdb.py",
> > line 1754, in _run
> > self.run(target.code)
> > File
> > "/nix/store/f2krmq3iv5nibcvn4rw7nrnrciqprdkh-python3-3.12.9/lib/python3.12/bdb.py",
> > line 627, in run
> > exec(cmd, globals, locals)
> > File "/home/karl3/projects/rep/rep/dict.py", line 394, in <module>
> > doc.update([[val,val]])
> > File "/home/karl3/projects/rep/rep/dict.py", line 349, in update
> > super().update(keyhashitems())
> > File "/home/karl3/projects/rep/rep/dict.py", line 164, in update
> > assert int.from_bytes(keyhash[:hashbytes], 'big') >> hashshift == newidx
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > AssertionError
> >
> >
> >
> > for keyhash, item in keyhashitems:
> >
> > assert item
> > != self._sentinel
> >
> > byteidx =
> > int.from_bytes(keyhash[:hashbytes], 'big')
> >
> > newidx = byteidx >> hashshift
> >
> >
> > if self._capacity > 0:
> >
> > # this
> > block checks for collision with previous stored values
> >
> > if capacity > self._capacity:
> >
> >
> > superidx = int.from_bytes(keyhash[:self._hashbytes],
> > 'big') >> self._hashshift
> > else:
> >
> >
> > superidx = newidx
> >
> >
> > place = self.array[superidx]
> >
> > if place !=
> > self._sentinel:
> >
> > collision = self._key(place)
> >
> >
> > if collision != keyhash:
> >
> > assert
> > superidx == int.from_bytes(collision[:self._hashbytes], 'big') >>
> > self._hashshift
> > updates[newidx] =
> > [collision, place, False]
> >
> > # this separated approach to checking for collisions
> > allows for accepting
> > #
> > batched data that ends up containing hash collisions solely within
> > itself
> > placing =
> > updates.get(newidx)
> > if placing is not None:
> > collision, place, is_new = placing
> > while newidx == int.from_bytes(collision[:hashbytes],
> > 'big') >> hashshift:
> > capacity <<= 1
> > expansion <<= 1
> > #spread += 1
> > #hashbits = self._hashbits + spread
> > hashbits += 1
> > hashbytes = (hashbits+7) >> 3
> > hashshift = (hashbytes << 3) - hashbits
> > byteidx = int.from_bytes(keyhash[:hashbytes], 'big')
> > newidx = byteidx >> hashshift
> > assert capacity == (1 << hashbits)
> > new_updates = {}
> > for keyhash, item, is_new in updates.values():
> > if is_new:
> > newnewidx =
> > int.from_bytes(keyhash[:hashbytes], 'big') >> hashshift
> > assert newnewidx not in new_updates
> > new_updates[newnewidx] = [keyhash, item, True]
> >
> > updates
> > = new_updates
> > assert newidx not in updates
> > assert int.from_bytes(keyhash[:hashbytes], 'big') >>
> > hashshift == newidx
> > updates[newidx] = [keyhash, item, True]
> >
> >
> > it looks like the problem is that keyhash is shadowed in the loop at the
> > end :D
shadowing is when a variable is used in an inner scope with the same
name as an outer scope. many languages have strictly scoped blocks but
python is not one of those languages allowing occasional mistakes like
this. keyhash is used in an outer scope. i then use it as a loop
variable, which changes its value in the outer scope.
usually a shadowed variable would make the opposite crash -- where in
the inner loop, the inner value is used as if it were the outer. here,
since it's python, the crash is from the outer value being used after
the inner loop has finished. [...
>
> now i've got this:
> File "/home/karl3/projects/rep/rep/dict.py", line 220, in content_generator
> assert superidx * expansion + subidx ==
> int.from_bytes(dbg_keyhash[:hashbytes], 'big') >> hashshift
>
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> AssertionError
> Uncaught exception. Entering post mortem debugging
> Running 'cont' or 'step' will restart the program
> > /home/karl3/projects/rep/rep/dict.py(220)content_generator()
> -> assert superidx * expansion + subidx ==
> int.from_bytes(dbg_keyhash[:hashbytes], 'big') >> hashshift
> (Pdb) p item
> b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
>
> but it's just because item is a sentinel which shouldn't hash to its
> index, can check for that
