cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
Smigwell has quit [Remote host closed the connection]
tsaka__ has quit [Remote host closed the connection]
<antocuni>
I thought we were talking about tests which were failing because we mix cpython's msvcrt runtime with our own
<antocuni>
but if it's a lib-python test, then it's run on top of a translated pypy, so there shouldn't be any mixture
<mattip>
right. So I want to fix the failing libpython test, so I write a untranslated one,
<mattip>
and it can't pass because in the mean time what is considered an invalid format string by msvc changed
<antocuni>
ah, I see
<mattip>
same thing with thread tests: we can't test locks because we allocate some via the host python and some via compiled code
<mattip>
so, like many of my branches, I got to 90% but there is still something wrong that is taking 90% of the time to fix
<mattip>
or even find
<antocuni>
the big question is: do we want to support this? It might be fine to just say that we can't run those tests untranslated
Smigwell has joined #pypy
<mattip>
well, how do we fix things then?
<mattip>
winioconsole is stuck, the py3.7 branch cannot run lib-python tests to completion without some strange pytest crash, ...
<mattip>
there are weird unicode locale conversion errors ...
<mattip>
another approach that might be easier would be to compile cpython2.7 with the same msvc compiler
<antocuni>
or to run untranslated tests with a compiled pypy
<mattip>
anyhow, this is getting kind of long. Thanks for listening
<mattip>
what I would _really_ like to do is release a 7.3.2 with a prototype hpy
<antocuni>
mattip: I fear I wasn't terribly helpful, though. But indeed, if doing the "correct" thing proves to be too hard or consume too much time, the workaround of recompiling cpython sounds like a good compromise to me
<antocuni>
at the moment, the pypy hpy is not compatible with the github one, but I plan to fix it soon
<cfbolz>
(note that for 64 bit windows we need a recompiled CPython to translate anyway)
<cfbolz>
*we will need
<mattip>
right, that's what got me thinking along those lines
* antocuni
--> lunch
ronan has quit [Remote host closed the connection]
ronan has joined #pypy
ronan is now known as Guest41816
LordKalma has joined #pypy
<LordKalma>
hey guys! I saw on the cffi docs to reach you guys here.
<LordKalma>
I'm having this funny problem. I have this library that doesn't compile with MSVC because MS can't implement C99 properly... Anyway, I built a shared library, on windows, with mingw, and now I was trying to make a cffi interface
<LordKalma>
I managed to make setuptools use mingw, all okay
<LordKalma>
but funny thing is that cffi wants the dll to be a .so
Smigwell has quit [Quit: Smigwell]
<LordKalma>
in fact, it's more bizarre than that. If I put the library on set_source as libraries=["mylib"], it wants the dll, having the file as .so yields a linker error
<LordKalma>
but then, after compilation, it gives an error, and a dependency walker shows me that the pyc wants a .so file
<LordKalma>
if I declare the .so file (just renaming the dll) as extra_objects in set_source, all is good
<LordKalma>
but it's weird to have a .so on windows...
jacob22 has joined #pypy
jacob22_ has quit [Read error: Connection reset by peer]
<antocuni>
side note: at some point, we need to think about a medium term strategy for compiling rpython code, because I fear that eventually python2.7 is going to stop working/compiling
<antocuni>
so we could end up maintaining our own stripped down version of cpython2.7, with just enough modules to complete a translation
LordKalma has left #pypy ["Leaving"]
<mattip>
it seems we are fast approaching that for windows anyway
<ronan>
antocuni: may I point out that this is completely bonkers? ;-)
Rhy0lite has joined #pypy
jthistle has joined #pypy
<antocuni>
ronan: why?
<antocuni>
if I do "hg branches" I see these two lines:
<ronan>
antocuni: yes, but that's only a test_ztranslation issue
<ronan>
I'm not too interested in fixing it, tbh
<antocuni>
ok, but it needs to be fixed anyway :). test_ztranslation is too useful, we can't remove it
<antocuni>
anyway, I think it's better if I go back working on the main hpy repo, to avoid conflicts. Moreover, I'm working on hpy only today, then I'll do some days of vacation
<ronan>
test_ztranslation becomes a complete mess as soon as we import from pypy.objespace.std
<ronan>
Nowadays, I'd rather rely on the rpython check in Heptapod CI
<antocuni>
as you prefer, but I speak of experience: I originally introduced test_ztranslation because fixing _hpy_universal translation was too hard without it
<antocuni>
moreover, the time invested in fixing it will pay for itself in the future, whenever we will need to fix translation again
<antocuni>
anyway, what is the error you get with HPyDef_Slot? The cast looks innocent to me
<ronan>
the problem is that it only exists in llapi.py, not in the actual generated C source
<antocuni>
you should be able to fix it by adding the relevant code to post_include_bits
<antocuni>
and while you are at it, maybe rename to _pypy_HPyDef_Slot or so
marky1991 has quit [Ping timeout: 240 seconds]
<ronan>
I think the cast is UB, but it's not like we really care
<antocuni>
UB?
<ronan>
undefined behaviour
<antocuni>
ah yes, sure
<ronan>
I was thinking of adding it to a header in src/, is post_include_bits better?
<antocuni>
a header is fine as well; currently we use post_include_bits for similar stuff, but indeed if it grows too much, a header is better
<mattip>
so now there is a vs2019 project for cpython 2.7
<mattip>
and I have a python2.7 built with it. I think I will try to run the buildbot worker with it, let's see what happens
<arigato>
mattip: as antocuni mentioned, maybe we should just use pypy2.exe instead of cpython to run untranslated tests
<mattip>
sure, even the JIT ones?
<arigato>
it doesn't matter, but why not?
<mattip>
I thought there was some issues, confusion with logs and stuff
<arigato>
at least it should be translated with the same version of the compilers that we test with (unless it was compiled on a different machine with a different configuration=
<mattip>
like what happens when you are running untranslated tests and you set PYPYLOG
<arigato>
if there are issues, it would be better to fix them than attempt to use a non-standard cpython (even though that's what we're doing right now for win64)
<arigato>
"ah"
<mattip>
but maybe that is only a handful of tests
<arigato>
or it can be worked around, e.g. by changing the name "PYPYLOG" somewhere
<mattip>
"translated with the same version of the compilers that we test with" - was what I was trying to do with the msvcrt branch
<mattip>
but I can't get it to work
<mattip>
or maybe you meant something else
<arigato>
I meant, making sure the host version and the tests are using the same clib and there is only one clib in total around
<arigato>
but I meant it by making sure the host python was a pypy translated using the same machinery as the one installed on the test machine
<arigato>
so there would never me two different clibs
<arigato>
s/me/be
<mattip>
got it
<mattip>
so then we either use a cpython or pypy built with the target compiler
<mattip>
which is more likely to work sustainably? I guess you are saying pypy
<arigato>
well maybe it's not working because we probably can't use some official pypy2.exe if it was built on another machine
<arigato>
so I have no idea
<arigato>
if building our own cpython is easy, go with it, I guess
<mattip>
took 10 minutes, once I had the pieces installed (visual studio was needed to upgrade the project files)
<arigato>
it's necessary right now for win64, but soon we'll have the same choice there, of either continuing to use our hackish cpython6464 or always using pypy2-win64.exe
<arigato>
OK
<arigato>
in fact we have the choice right now, because we can build a pypy2-win64.exe, just not with all extra modules yet
<mattip>
let me switch to win32 pypy2 and kick the rpython build bot, lets see how many new failures there are