cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | the secret reason for us trying to get PyPy users: to test the JIT well enough that we're somewhat confident about it
moei has joined #pypy
commandoline has quit [Ping timeout: 252 seconds]
commandoline has joined #pypy
igitoor has quit [Ping timeout: 252 seconds]
kipras has quit [Ping timeout: 252 seconds]
igitoor has joined #pypy
kipras has joined #pypy
igitoor has quit [Changing host]
igitoor has joined #pypy
oberstet has joined #pypy
alexband has joined #pypy
alexband has quit [Read error: Connection reset by peer]
alexband has joined #pypy
alexband has quit [Read error: Connection timed out]
alexband has joined #pypy
jcea has quit [Quit: jcea]
simpson has quit [Ping timeout: 260 seconds]
dddddd has quit [Remote host closed the connection]
simpson has joined #pypy
il_ratto has quit [Ping timeout: 245 seconds]
nopf has quit [Ping timeout: 240 seconds]
il_ratto has joined #pypy
nopf has joined #pypy
danieljabailey has quit [Ping timeout: 240 seconds]
danieljabailey has joined #pypy
speeder39 has quit [Quit: Connection closed for inactivity]
pjenvey1 has joined #pypy
pjenvey has quit [Read error: Connection reset by peer]
danchr has quit [Ping timeout: 268 seconds]
danchr has joined #pypy
fryguybob has quit [Ping timeout: 246 seconds]
marmoute has quit [Ping timeout: 260 seconds]
marmoute has joined #pypy
fryguybob has joined #pypy
forgottenone has joined #pypy
LarstiQ has quit [Ping timeout: 246 seconds]
LarstiQ has joined #pypy
mdroid has joined #pypy
mdroid has quit [Remote host closed the connection]
solarjoe4 has joined #pypy
tayfun26 has joined #pypy
Sove has joined #pypy
Sove has quit [Remote host closed the connection]
solarjoe4 has quit [Quit: Leaving]
jamesaxl has joined #pypy
forgottenone has quit [Quit: Konversation terminated!]
forgottenone has joined #pypy
Guest30013 has joined #pypy
Guest30013 has quit [Remote host closed the connection]
forgottenone has quit [Quit: Konversation terminated!]
solarjoe4 has joined #pypy
forgottenone has joined #pypy
antocuni has joined #pypy
arigato has joined #pypy
realitix has joined #pypy
<mattip>
bitbucket ui is really annoying. Nowhere does it tell you to sign in to interact with the site, markdown doesn't work, issues are hidden from the project's front page ...
<kenaan>
arigo default 99c1daa566cf /pypy/module/select/: Define select.PIPE_BUF only when it exists on CPython
<antocuni>
apparently, git has "won", so I'd not be surprised if eventually we need to migrate anyway
<LarstiQ>
antocuni: it has been like that for years though, so I'm not expecting any change in mercurial usage in the near to midterm future
<antocuni>
true enough
<arigato>
until we find a git extension that give true branches, I'm -1
<arigato>
I regret having picked git on the VR project whenever I need to dig into the history
solarjoe4 has quit [Quit: Leaving]
<arigato>
(not really a critic, just saying that my own style doesn't work that well without real branches)
<antocuni>
you mean that a single commit doesn't remember in which branch it was added?
<arigato>
yes, and also things like "git bisect": suddenly you'll be bisecting the internals of some branch, which you don't always want to
<arigato>
I tend to make "in-progress" commits, on branches
<antocuni>
yeah, I agree that named branches is the single hg feature I miss in git
<arigato>
I still like your hg log layout, I think I removed a few colors but not the bright red branch name :-)
<antocuni>
:)
<antocuni>
the curious thing is that nobody else seems to care much about named branches; even in hg, they introduced bookmarks to be more similar to git branches
<arigato>
no clue, different style I guess
hurify1 has joined #pypy
<mattip>
the gitlab support will be bilingual - you can use either mercurial or git it seems, if I read correctly
<arigato>
cool
<arigato>
mattip: fwiw, on bitbucket, you need to remain logged to avoid some of the issues you're having
<mattip>
not so much for me, but it is a barrier to new users
<arigato>
ah
<arigato>
ok
<mattip>
just rambling now, but that is why I don't understand CPython's move to git but leaving the issue tracker elsewhere,
<arigato>
shrug
hurify1 has quit [Ping timeout: 244 seconds]
<mattip>
if the goal is to encourage engagement, usually people start with issues then move to pull requests
<mattip>
although with so many open issues maybe they don't want to encourage more :)
arigato has quit [Quit: Leaving]
arigato has joined #pypy
waldhar_ has quit [Ping timeout: 272 seconds]
antocuni has quit [Ping timeout: 246 seconds]
* cfbolz
waves
<arigato>
hi
* arigato
tries *again* to set up pypy on windows, and *again* it fails with an obscure compilation error. I'm impressed how it fails every single time I try, differently
<mattip>
arigato: it would be good to solve the "clean translation on windows" problem. Microsoft breaks compatibility every time they update the visual studio download
<mattip>
so what worked two years ago now breaks
<arigato>
I'm far too scared to go there
<arigato>
right now I'm righting with build_cffi_modules.py
<mattip>
right, it cannot install setuptools since it cannot download pip
<arigato>
for no reason I can see, it suddenly decided to throw "unresolved external symbol" errors from libeay32.lib
<mattip>
ahh, that is new. Is it using the wrong compiler/linker?
<arigato>
as far as I can tell, I changed exactly nothing about it
<mattip>
translation uses our custom rpython/translation/platform/windows tool to find the compiler, I don't think build_cffi does
<arigato>
I should have only one compiler installed anyway
<arigato>
one of the missing symbols is "_sscanf", that doesn't make much sense
<mattip>
hmm. The get_externals script is supposed to update ./externals with the proper libs for this pypy2/3 and msvc versions
<arigato>
uh
<mattip>
what hg id do you see there
<arigato>
ok, I didn't actually know about it, I'm using manually-installed stuff from long ago
<mattip>
I wonder if I documented that properly
<mattip>
I guess not if you don't know about it
<arigato>
if it's written in the pypy.org/windows page, then it's probably fine
<arigato>
I admit I don't re-read this page whenever I have an issue :-)
<njs>
mattip: I think mostly the issue is that the old way of doing PRs ("please upload a unified diff as an attachment") was so dreadful that it was the most urgent thing
<mattip>
njs: thanks. Is python using zulip instead of IRC these days? (I see that is where discusison on the PEP happens
<mattip>
)
<njs>
mattip: I'm not sure... they do have a zulip install and it gets used a bit, but I doubt anyone has shut down the IRC channel either...
<mattip>
arigato: I kicked a build of unicode-utf8-py3 on win32, let's see if a) it translates, b) it compiles that regexp
<tos9>
they experimented with a couple of things
<arigato>
mattip: in general, I fear there's a lack of tests for the >65535 unichars in all kinds of situations
arigato has quit [Quit: Leaving]
alexband has quit [Ping timeout: 246 seconds]
lritter has joined #pypy
oberstet has quit [Ping timeout: 252 seconds]
jcea has joined #pypy
jcea has quit [Ping timeout: 252 seconds]
jcea has joined #pypy
jamesaxl has quit [Read error: Connection reset by peer]
jamesaxl has joined #pypy
oberstet has joined #pypy
jamesaxl has quit [Read error: Connection reset by peer]
jamesaxl has joined #pypy
antocuni has joined #pypy
<kenaan>
mattip py3.5 abeffc967e2e /pypy/module/_csv/: make csv bytes error more compatible
<kenaan>
mattip unicode-utf8-py3 68f116b03eb4 /pypy/module/: fixes for failures in array, _pypyjson
<kenaan>
mattip unicode-utf8-py3 a0b258c2d6e0 /pypy/module/_csv/: merge py3.5 into branch
<kenaan>
mattip unicode-utf8-py3 5fdece607d46 /pypy/module/_csv/interp_reader.py: use utf8 iterator in _csv.reader
<kenaan>
mattip unicode-utf8-py3 7dd0a62dbf67 /pypy/module/cpyext/: use unicodehelper methods instead of rffi calls
realitix has quit [Ping timeout: 252 seconds]
realitix has joined #pypy
realitix has quit [Remote host closed the connection]
realitix has joined #pypy
domenkozar has quit [Ping timeout: 250 seconds]
domenkozar has joined #pypy
realitix has quit [Quit: realitix]
waldhar has joined #pypy
marky1991 has joined #pypy
marky1991 has quit [Ping timeout: 272 seconds]
vstinner has joined #pypy
<vstinner>
hello. discussions on the C API are ongoing on the capi-sig, but I don't see PyPy developers involved. i would love to see you involved, since we have a good experience in issues caused by the current ("old") C API
Khayman has joined #pypy
TheAdversary has quit [Disconnected by services]
Hasimir- has joined #pypy
TheAdversary has joined #pypy
Hasimir| has joined #pypy
Hasimir has quit [Ping timeout: 245 seconds]
Khayman has quit [Ping timeout: 245 seconds]
<vstinner>
my current question is if Py_TYPE() returning a borrowed reference is an issue or not
<simpson>
vstinner: I'm merely an RPython user, but after reading some of the docs you've got up on Github, I feel like there's some Blub in the way. IMO any new API has to be convincingly better than https://cffi.readthedocs.io/en/latest/embedding.html
Hasimir- has quit [Ping timeout: 245 seconds]
<vstinner>
simpson: i don't want to write a new API from scratch but make "small changes" to the existing C API
<simpson>
Case in point, trying to reflect GC'd refs into C is futile, because C simply doesn't have the ability to enforce the GC's invariants. It's much easier to pull stuff into Python and wrap it there.
<vstinner>
simpson: my problem is the long tail of C extensions on PyPI which are written with the C API
<simpson>
vstinner: Sure, and so my answer (even though I'm not a core committer) to capi-sig is "No, I already gave up on getting good things from CPython"
<simpson>
But I'm happy to join if you need a voice which says nothing other than "Deprecate C/API"
<vstinner>
"No, I already gave up on getting good things from CPython" i'm sorry about that :-(
<vstinner>
simpson: "Deprecate C/API" is not part of _my_ plan
TheAdversary has quit [Disconnected by services]
Hasimir has joined #pypy
<vstinner>
simpson: but it's the most important point of the PyPy plan for 10 years, no? is it successful?
<vstinner>
:)
<simpson>
Don't be. If every pork chop were perfect, we'd never have sausages; I had to give up on Python in order to get Monte off the ground.
Khayman has joined #pypy
<vstinner>
i know that the C API sucks, the thing is that people use it
<simpson>
Yeah, PyPy has this weird dream where they expect *Python* users to write code in *Python*~
<vstinner>
simpson: i know. and now people argue about using Cython
<vstinner>
cffi, Cython, C API. pick your favorite one :) in the meanwhile, CPython has to maintain the C API
<vstinner>
the C API is part of Python, deal with it ;)
<vstinner>
i offer your an opportunity to make the C API less bad :)
<vstinner>
simpson: oh ok, nice. but for my case, i only care about Python :)
<simpson>
Sure. Hopefully some folks around here are motivated to participate in capi-sig.
Hasimir has quit [Ping timeout: 246 seconds]
Khayman is now known as Hasimir
TheAdversary has joined #pypy
<simpson>
vstinner: It would be nice if, given that some 98% of the community either chose C/API over PyPy or chose something which chose for them, a PEP existed which opined whether the C/API were, in fact, a blessed part of Python's API.
<simpson>
PEP 8 exists; there's precedent for declaring that certain styles are (un)acceptable. No document that I know of states the PSF's explicit preference for Python, nor espouses a pure-Python or hybrid-Python-C viewpoint.
<vstinner>
simpson: i don't think that anyone should bless the C API. i hate it :)
<vstinner>
simpson: my long term plan is more to remove the C API :)
<simpson>
vstinner: Right, but then why doesn't your rationale or roadmap say that?
<vstinner>
simpson: it's more the long long term :)
<simpson>
vstinner: To be blunt, you mean that it's politically unsavory within PSF to hold this position?
<vstinner>
removing the C API is not part of http://pythoncapi.readthedocs.io/ because i prefer to only announce things that can be done myself in the mid-term :)
<vstinner>
simpson: removing the C API today means removing support for all C extensions. it means to kill Python, in short
<vstinner>
simpson: it just doesn't make any sense, today
<simpson>
I wager that, given a week of PTO (which I am willing to commit to!) I could produce a CPython build with no C/API.
<simpson>
Ah, sure. Yeah, so what you have to do is *announce your intent* first.
<LarstiQ>
vstinner: ah, but the pypy perspective I wager is that the fix is to deprecate it?
<vstinner>
many people told me that it's fine that Py_TYPE() returns a borrowed reference
<simpson>
For starters, a bad API is one which leaks abstraction details. C has no notion of borrowing!
<LarstiQ>
but maybe a layered approach makes sense
<vstinner>
LarstiQ, simpson : IMHO it doens't make any sense to drop support for C extensions. C extensions are part of Python. i don't think that it's worth it to discuss this point
<vstinner>
if you disagree, see how long it took to migrate from *Python* 2 to *Python* 3
<vstinner>
at the C level, any change will take much longer
<vstinner>
maybe my whole plan doesn't make any sense neither :)
<simpson>
Ha, yeah, that silliness is still happening. Just had to convince my employer last week that just because *CPython 2.7* is going away, doesn't mean that we have to rewrite all our code, since PyPy 2.7 support will be ongoing.
<fijal>
vstinner: hi
<fijal>
"IMHO it doens't make any sense to drop support for C extensions. C extensions are part of Python"
<fijal>
I think this is a bit why for us "it does not quite make sense to engage in discussions"
<fijal>
maybe "let's completely stop adding new C APIs"
<fijal>
and "let's deprecate useless C APIs" would be however a step forward
<vstinner>
fijal: hi. i would like to know which functions/features of the C API or C language are causing you the most troubles. i already listed some of your proposals at https://pythoncapi.readthedocs.io/bad_api.html
<fijal>
Py_INCREF/Py_DECREF
<vstinner>
right now, i'm focused on borrowed references. i expected that everybody would agree that they are bad, but it seems like people would like to make compromises for Py_TYPE().
<vstinner>
fijal: what do you propose to replace INCREF/DECREF?
<fijal>
cffi
<fijal>
or *anything in Python*
<fijal>
anyway, sorry, I'm being counter productive here
<fijal>
maybe mattip would have an opinion
<simpson>
fijal: FWIW I've found cffi to be a big boon to productivity! So I appreciate it. I especially appreciate that your proposal exists and works today on both CPython and PyPy!
<fijal>
99% of that is pointless to have as a C function
<vstinner>
fijal: "cffi" means "don't use the C API". hum, we restart the discussion from the start...
<vstinner>
fijal: "99% of that is pointless to have as a C function" i'm working on practical solutions to be able to make changes to the C API. that's why i'm here
<vstinner>
fijal: right now, it's not possible to change anything :)
<fijal>
ok, so what do you want from me?
<fijal>
"it's not possible to change anything", despite a lot of it being deprecated (?)
<simpson>
vstinner: Well, if the goal is to write C code and call it from Python, then cffi does that, right? So the only reason to avoid cffi would be because one doesn't want to rewrite piles of messy C? But rewriting piles of messy C *always* is unpleasant, I think.
<fijal>
but also "it's not possible to stop adding new pointless functions"?
<vstinner>
the key of my proposal is to propose two runtimes instead of one. one legacy runtime which implements the old C API which sucks, and one new runtime for C extensions ported to the new C API
<fijal>
and what would be in the new C API?
<simpson>
Can the new runtime have no C extensions, and no new C API, and in fact just be pure-Python?
<simpson>
But with cffi support for all the things one used to do?
<simpson>
IOW can we bless a community-wide effort to port to cffi or similar?
<fijal>
there is the issue of numerics
<vstinner>
simpson: the goal... oh god, it seems like my Rationale page sucks :-( i should reword it :-/
forgottenone has quit [Quit: Konversation terminated!]
<simpson>
vstinner: My proposal (trivially) satisfies your design goals!
<fijal>
vstinner: but seriously, since python 3 you guys have been adding tons and tons and tons of new stuff in C API
<vstinner>
simpson: " Can the new runtime have no C extensions, and no new C API, and in fact just be pure-Python? " hum. let's call it "PyPy" and see if it's successful ? ;-)
<vstinner>
fijal: i know, i know. i'm not here to discuss mistakes of the past :)
<fijal>
so IMO, it's a loosing battle - I don't see the point of arguing with anyone who thinks that PyUnicode_Translate is a good function to have in C
<fijal>
life is too short to fight over such things, I think
<fijal>
you guys decided C API is essential, which IMO decides that CPython will be the only interpreter ever with >1% of users
<simpson>
vstinner: ~ But more seriously, bring Cython into the CPython stdlib hood, and then bind it tightly to CPython's internals. Then there's no need to expose things just for Cython.
<fijal>
Cython or something more minimal
<vstinner>
fijal, simpson : when i wrote previously "i hate the C API, i want to remove it" in fact, my idea is more to remove most functions until you get the smallest set of functions just good enough to write a C extension
<fijal>
so yeah, IMO it's a death for Python at some stage and that's it - I think it'll eventually bite you in the ass
<fijal>
ok, but that's not happening, is it?
<vstinner>
for example, why do we have PyTuple_GetItem() when there is already PyObject_GetItem()?
<fijal>
the new functions are added with every single CPython release right?
<fijal>
so maybe "for sure no new functions" as a starting point?
<vstinner>
fijal: "I think it'll eventually bite you in the ass" my ass already hurts. why do you think that it would like to modify the C API?
<fijal>
I appreciate your enthusiasm (I really do)
<fijal>
but I do feel like someone arguing over every single piece of trash with a compulsive hoarder
<fijal>
(not you personally, just python-dev in general)
<vstinner>
fijal: so maybe "for sure no new functions" as a starting point? <= i'm not sure that it's doable. in my early experiments, i already added many functions: https://pythoncapi.readthedocs.io/new_api.html
<fijal>
well, then "no"
<fijal>
because most added functions can be replaced with a very simple one
<fijal>
like PyCall(PyObject_GetAttr(x, "translate"))
<fijal>
or something
<fijal>
"no" as in I don't see myself being a part of this discussion
<vstinner>
maybe adding PyTuple_GetItemRef() doesn't make sense, but what about Py_TYPE()? if we convert it to a function, Py_TYPE(obj) = type; becomes a compilation error. we need a new function, no?
<fijal>
but again, maybe mattip would have a more balanced opinion, I'm just explaining why I won't be a part in that discussion
<fijal>
note that we (painfully) implemented all of this mess, so for us it's a bit whatever
<simpson>
fijal: Ha, get out of my head, I was literally typing `PyCall("__getitem__", EMPTY_ARGSPEC)`
<fijal>
"stop adding new ones" is a good start
<vstinner>
fijal: "because most added functions can be replaced with a very simple one" i agree. but i'm not sure that we can go directly from the current state to your ideal API. first we need tooling to port code, document what is a bad API, etc.
<fijal>
I'll be dead before you guys get to any point
<simpson>
vstinner: Is the goal to convince core devs, or to convince numpy maintainers?
<simpson>
Hopefully core devs are reasonable folk, but every discussion with numpy folk I've had has been completely useless.
<fijal>
numpy mainteiners can move to cython
<fijal>
or cython-like thing
<vstinner>
simpson: convince everyone. core devs cannot go alone and force everybody to move on if there is no consensus
<vstinner>
simpson: i already started to talk with Nathaniel for the numpy part
<fijal>
vstinner: *ekhem*, they did with python3
<vstinner>
i understood that *nathaniel* would prefer to rewrite numpy into a different language, like Cython or something else
<vstinner>
but i'm not sure that he will succeed. many people have long term plan but no time :(
<vstinner>
fijal: i gav a talk about the migration to python 3. in fact, we learn from our mistakes ;)
<simpson>
Sure. I tried to rewrite PIL twice and failed twice. It's hard.
<vstinner>
i gave*
<vstinner>
simpson: PIL Python Imaging Library?
<vstinner>
simpson: converting all C extensions to Cython, cffi or anything else is a nice plan, but it will no work. it takes an unlimited time to contact each maintainer, send a patch, explain again why a change is needed, etc.
<vstinner>
i did it for python 3, and it's a painful task :-(
<vstinner>
fijal: instead of "stop adding new ones", i would prefer to say "don't add new functions if it's already possible do the same thing with existing functions"
<simpson>
vstinner: Oh, I just started rewriting it in pure Python. Because it's not worth *anybody's* time to write new C/API code.
<vstinner>
fijal: PyContextVar_Get() has been added to Python 3.7. i'm not sure that it's possible to implement it with existing functions. but it's also a matter of efficiency, sadly
<vstinner>
simpson: oh ok
<vstinner>
simpson: was it fast?
<vstinner>
fijal: "I'll be dead before you guys get to any point" lol, maybe :-) we can bet on that!
<simpson>
I had fundamental issues getting the API to line up because the C code was so abusive of C/API.
<simpson>
But sure, yeah, I'm happy to give you my "PyPy is fast" lesson: Rewrote a numeric kernel for a game server from numpy to array.array + PyPy. Got a 60x speedup on benchmarks and a 20x speedup on a load test.
<vstinner>
simpson: there is an ongoing discussion to modify the Python language to support positional-only argument :) that's still a big difference between C extensions and Python code
<LarstiQ>
simpson: is there a public writeup of that?
<simpson>
Folks have given feedback like "you didn't vectorize enough", but have failed to actually point out how that could be done.
<simpson>
The *actual* speedups are a little hidden, though; they're in a numeric kernel that was always pure-Python.
<vstinner>
fijal: at least, we agree on one people: there are many variants of the same functions in the C API wheras a single one would be enough. like PyTuple/PyList/..._GetItem() vs just PyObject_GetItem()
<simpson>
Yes, I think we're all agreed on that.
<antocuni>
simpson: please write a blog post about this numeric kernel thing
Khayman has joined #pypy
oberstet has quit [Ping timeout: 246 seconds]
<vstinner>
oh stupid me, PyObject_GetItem() expects key as a PyObject*, whereas PyTuple_GetItem() expects key a Py_ssize_t :-p it's not *exactly* the same API ;-)
TheAdversary has quit [Disconnected by services]
<antocuni>
vstinner: fwiw, I subscribed to the capi-sig ML; maybe I'll be able to add some contribution which is a bit more constructive than what you got here :)
<vstinner>
antocuni: more constructive than what you got here" i don't see what you mean ;-)
TheAdversary has joined #pypy
<antocuni>
in the sense that reading the traceback, it seems that you got mostly non-constructive comments here on #pypy (like "drop the C API", which I would also like to happen but I agree it's impossible)
Hasimir has quit [Ping timeout: 246 seconds]
Khayman is now known as Hasimir
<vstinner>
antocuni: "I agree it's impossible" yep
<vstinner>
i have to go, see you maybe later ;-)
<antocuni>
bye
oberstet has joined #pypy
marky1991 has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
marky1991 has quit [Ping timeout: 252 seconds]
antocuni has quit [Ping timeout: 240 seconds]
abrown has joined #pypy
nunatak has joined #pypy
adamholmberg has joined #pypy
marky1991 has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
abrown has quit [Ping timeout: 252 seconds]
adamholmberg has quit [Remote host closed the connection]
adamholmberg has joined #pypy
adamholmberg has quit [Ping timeout: 240 seconds]
marky1991 has quit [Ping timeout: 240 seconds]
marky1991 has joined #pypy
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
nunatak has quit [Quit: Leaving]
marky1991 has quit [Remote host closed the connection]
marky1991 has joined #pypy
adamholmberg has joined #pypy
Taggnostr2 has joined #pypy
arigato has joined #pypy
DIRT has joined #pypy
<kenaan>
arigo default b9bbd6c09333 /: I remember now why we moved away from using CPython's definition of constants. Now translation (and own tests) pas...
antocuni has joined #pypy
adamholmberg has quit [Remote host closed the connection]
arigato has quit [Quit: Leaving]
oberstet has quit [Quit: Leaving]
adamholmberg has joined #pypy
dddddd has joined #pypy
marky1991 has quit [Ping timeout: 252 seconds]
lazka has joined #pypy
<lazka>
vstinner, re bad API, PyModule_AddObject() steals a ref in case it succeeds, but doesn't in case of an error.
<lazka>
(and it's not documented)
<vstinner>
lazka: oh... i *hate* such aweful APIS :-(
<vstinner>
lazka: i found some other functions like that, but not public functions
forgottenone has joined #pypy
<lazka>
there is a general lack of reference counting info in the docs, I always have to grep cpython
<vstinner>
lazka: i agree
adamholmberg has quit [Remote host closed the connection]
<vstinner>
and the C API has no unit test :)
<lazka>
whhaaaa.. ha :)
<antocuni>
vstinner: there are cpyext tests :)
<antocuni>
you can run them even on cpython
<vstinner>
lazka: i started to write unit tests for the C API
<vstinner>
antocuni: how did you write tests for the C API?
<vstinner>
cfbolz: what would you use to write unit tests for the C API?
<cfbolz>
python ;-)
<antocuni>
vstinner: look e.g. at pypy/module/cpyext/test0/test_methodobject.py
<antocuni>
we have python tests, small snippets of C code inside the tests (as a string), and a way to automatically build and import a module containing the C code
<vstinner>
antocuni: hum, these tests depend on pypy and rpython
<antocuni>
yes, but if you run them with py.test -A, they are run on top of CPython
<vstinner>
antocuni: oh nice
<antocuni>
it is surely possible to write them in a way in which the core tests does not depend on pypy/rpython, but can be imported in pypy
<antocuni>
the magic of -A is defined inside conftest.py and pypy.tool.pytest.apptest: in practice, you can write tests which are normally run inside the pypy interpreter (running itself on top of cpython), or directly on top of the hosting interpreter if you pass -A
<vstinner>
antocuni: ok
<antocuni>
this applies both to cpyext and non-cpyext tests
<antocuni>
I think it would be very good to have C API tests which are shared between pypy and cpython
<vstinner>
antocuni: sure
dmalcolm has quit [Quit: Leaving]
lcollini has joined #pypy
<antocuni>
vstinner: I replied to your email about borrowed references; I don't know if this is what you wanted to say, but I highlighted something which is surely a problem for pypy
<vstinner>
antocuni: it's not really an issue, once it's written down in https://pythoncapi.readthedocs.io/ so later i can point to it ;)
<vstinner>
antocuni: i recall that armin reported *dozen* of super tricky bugs caused by borrowed references ;)
<antocuni>
in this precise case, they use PyMapping_GetItem which returns a new reference; but this happened because at some point in the past they used PyDict_GetItem, and then switched to PyMapping_*
<vstinner>
cute _PyType_Lookup API: /* Internal API to look for a name through the MRO. This returns a borrowed reference, and doesn't set an exception! */
<vstinner>
antocuni: hehe
<vstinner>
antocuni: IHMO the main point of borrowed references is that PyPy has no PyObject at all
<antocuni>
well, we kind of do have PyObject*, just that they are build lazily and attached to W_Root
<antocuni>
but the problem is when you don't have any W_Root at all
<antocuni>
it seems I should include a section about borrowed references as well :)
<vstinner>
antocuni: nice for the article
<vstinner>
antocuni: one of my motivation for rework the C API is to make PyPy faster (indirectly)
<vstinner>
antocuni: i decided to really work on this C API project while watching Ronan Lamy’s Talk "
<vstinner>
Adventures in compatibility: emulating CPython's C API in PyPy
<vstinner>
"
<vstinner>
emulating CPython design issues is just insane :-(
<antocuni>
vstinner: as a general advice on your nice document, I'd add more examples on why this or that change would make life easier for other implementations (pypy being the main example, probably). It is easier for people to understand things when they see an example of a concrete problem, instead of thinking in abstract terms
<cfbolz>
vstinner: is that talk recorded?
<antocuni>
cfbolz: it was at last europython
<lazka>
vstinner, what would have saved me some debugging time when porting to pypy is if PyType_Ready() would warn in case a base type isn't finalized. because with pypy that crashes