cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<antocuni>
wow, I just discovered that it exists a thing called "PseudoHighLevelCallable"
<antocuni>
which doesn't seem to be used anywhere, but the name is very cool :)
<LarstiQ>
now what could that mean? :)
<antocuni>
"""A gateway to a low-level function pointer. To high-level RPython
<antocuni>
code it looks like a normal function, taking high-level arguments
<antocuni>
and returning a high-level result.
<antocuni>
"""
<antocuni>
but don't ask me what *actually* means :)
<LarstiQ>
that sounds like a transparent python-to-c bridge?
<LarstiQ>
ish
<antocuni>
no I think it's on another level
<LarstiQ>
a high one? ;)
<antocuni>
in rpython a "high level function" takes things like rpython lists and strings. A low level function takes things like "struct _rpy_str { char *content; int lenght}"
<LarstiQ>
right
<antocuni>
so the same function can be seen with two different rpython types. I think this was meant to "level up" if you start with a low-level function, but I'm not sure what it was used for
<antocuni>
probably something in the v1 JIT, since according to hg blame it was introduced on 2006-08-24
<exarkun>
I wonder if anyone knows of any tricks I might employ to make it fail _more_? Right now I've only seen it on Tahoe-LAFS' PyPy CI job, and only once in a while. That makes it hard to be confident I've actually fixed it.
<simpson>
Maybe could patch it with worst-case behavior; is WeakValueDictionary still correct if its .__contains__() pops the key from itself and returns True?
<exarkun>
Hm yea I guess that's a possibility. Or maybe a not _quite_ worst-case but more realistic version? `__contains__` determines its result, the forces a full gc run, then returns it? I don't know if that would actually reliably provoke this case though.
<exarkun>
I did finally reproduce it one time on my laptop after 455 test runs.
<cfbolz>
exarkun: it's wrong because the dict entry disappears between the two lines?
<exarkun>
Oh right, sorry, yes, the way it fails is with a KeyError on the 2nd line
<exarkun>
And note it's not a regular dict but a WeakValueDictionary
<cfbolz>
the weak*dicts on pypy are really a bit annoying
<exarkun>
certainly they're a bit different than the CPython versions. but I remember being pretty annoyed with the CPython versions on more than one occassion. :)
<cfbolz>
exarkun: yes, in cpython it's "mostly nice, and sometimes really strange"
<cfbolz>
exarkun: I'd be tempted to go with simpson's suggestion, your idea is more realistic but sounds like it could lead to flaky or slow tests
<exarkun>
A complicating factor (applying to both ideas mentioned) is that the test suite is really complicated and I'm not actually convinced I can poke a different kind of dict into the implementation :/
<exarkun>
(Or, I'm sure I can, just not sure if it will cost $8000 to do so)
<exarkun>
If I can reproduce the problem locally about 30 times then I might call that statistically significant and I can validate the fix that way...
<exarkun>
I think what I originally had in mind when I asked the question is whether there was some `pypy -Xmakeweakrefsdowhatiwant ...` I could use to get closer to determinism without code changes
<exarkun>
Oh, well, as a temporary hack, if I just jam a gc.collect() right between the two lines then my average runs-until-failure goes from >700 to 1
<cfbolz>
exarkun: sorry, was afk. my suggestion was to write a very targetted regression test, that monkeypatches the dict implementation and then fix the code. I understand that it doesn't really help you in fixing the other potential problems with weak dicts
<exarkun>
cfbolz: Hm, yea, that is probably a pretty good suggestion.
<cfbolz>
:-)
<mattip>
nulano: around? Feel like taking a look at the pypy.module._rawffi failures on the win64-py3.7 branch?