cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<kenaan>
antocuni hpy-ctypespace ed73b0a1ed73 /: close branch which will be merged into hpy
<kenaan>
antocuni hpy 3239497d9665 /: Merge branch hpy-ctypespace. Use CTypeSpace to declare HPy types I think that the final result is much easier to re...
<kenaan>
antocuni hpy cf17a3d5456f /pypy/module/hpy_universal/: update the support machinery to take the new argument extra_templates; add test files for all the new hpy tests. Upd...
<kenaan>
antocuni hpy e2ab328b16a8 /pypy/module/hpy_universal/: use the updated values for HPy_METH_*
<kenaan>
antocuni hpy d8aab0b1b105 /pypy/module/hpy_universal/handles.py: add one more builtin singleton
<kenaan>
antocuni hpy 88722810d26d /pypy/module/hpy_universal/_vendored/test/support.py: update to pyhandle/hpy 2aec7ae
<kenaan>
antocuni hpy 64a75c77d1cc /pypy/module/hpy_universal/: finally! Collect all the api functions automatically and stick them into the context. We need to make this RPython, ...
<kenaan>
antocuni hpy 961b3a276537 /pypy/module/hpy_universal/: fix translation for the part which was broken by 64a75c77d1cc. Translation is still broken because of other problems...
rubdos has quit [Remote host closed the connection]
lritter has joined #pypy
squeaky_pl has joined #pypy
<squeaky_pl>
if you are looking for a simple c python extension to make tests on there is also google's brotli (web tailored compression algo that is used a lot these days) implementation https://github.com/google/brotli/blob/master/python/_brotli.cc. It has only a couple of functions.
bendlas has quit [Remote host closed the connection]
extraymond[m] has quit [Read error: Connection reset by peer]
edd[m] has quit [Write error: Connection reset by peer]
agates[m] has quit [Read error: Connection reset by peer]
<squeaky_pl>
I have a question about pyhandle and other languages that are not C, you seem to rely a lot on static inline which makes it less useful for languages like Rust. Are you gonna address this eventually?
squeaky_pl has quit [Ping timeout: 240 seconds]
forgottenone has quit [Read error: Connection reset by peer]
forgottenone has joined #pypy
dddddd has joined #pypy
squeaky_pl has joined #pypy
squeaky_pl has quit [Read error: Connection reset by peer]
squeaky_pl has joined #pypy
<antocuni>
squeaky_pl: thank you for the brotli link, although we can't port to HPy yet because we don't support declaring types for now
<antocuni>
squeaky_pl: static inlines are used only to have a nice C API on top of the "official" ABI. The ABI relies on the fact that we pass the HPyContext around, and all the calls are done by invoking a function pointer which is stored on it
<antocuni>
(one different function pointer for every API function, of course)
<squeaky_pl>
antocuni, thanks, I actually found the answer just minutes ago looking at the issues, there was Rust discussion there
CrazyPython has joined #pypy
<antocuni>
so, if RustPython wants to implement HPy, it "just" needs to provide an HPyContext whose pointers point to functions implemented inside the RustPython runtime
<antocuni>
then of course we also have the opposite direction: what is one wants to write an HPy extension in Rust? Here I'm not an expert, but I think it's probably possible to write a rust library which expose a nice-to-use API which is translated into calls such as ctx->ctx_NumberAdd(ctx, h_x, h_y)
<squeaky_pl>
yes, from my limited Rust knoledged, I think it would be putting a Rust struct with repr(C) layout annotation and putting pointers there with some unsafe code
edd[m] has joined #pypy
extraymond[m] has joined #pypy
agates[m] has joined #pypy
bendlas has joined #pypy
altendky has joined #pypy
<antocuni>
they way hpy is being designed makes it possible to write a nice C++ API as well
<antocuni>
e.g., I guess it should be possible to write some kind of "smart handles" which uses the RAII pattern to automatically close a handle when it goes out of scope
<antocuni>
but I admit that non-C extensions are not the highest priority for now
<squeaky_pl>
Yeah, same for Rust with drop implementation.
CrazyPython has quit [Read error: Connection reset by peer]
<mattip_>
antocuni: we updated pyrepl and fixed some unicode bugs, so issue 3123 and 3124 might not fail on HEAD
<mattip_>
also, it may depend on your LOCALE
<antocuni>
could be. It was discovered by some guy at gambit, then I could reproduce also on my machine
squeaky_pl has quit [Read error: Connection reset by peer]
<antocuni>
but maybe we have the same LOCALE :)
squeaky_pl has joined #pypy
jvesely has joined #pypy
* antocuni
tries with a nightly build
<antocuni>
mattip_: I confirm the bugs are still there even with the nightly of rev e5d59224adb8
<mattip_>
thanks. It seems to be some interaction with `LANG=`, since without that they pass
<antocuni>
yes; I checked sys.getdefaultencoding() but it always says 'utf-8', even with LANG=
<antocuni>
but I didn't investigate further because I am happily in hpy land right now :)
<mattip_>
LANG seems to affect sys.getfilesystemencoding
<mattip_>
it is nice to see progress on hpy
<antocuni>
but I think that stdout should be encoded by getdefaultencoding(), not getfilesystemencoding(), shouldn't it? (I might be wrong, I never manage to remember all these messy details :))
squeaky_ has joined #pypy
<antocuni>
mattip_: re hpy: yes, I hope to be able to run ujson_hpy on pypy soon. I'm curious to see what is the performance
<mattip_>
the centos-6 based pypy built in the docker runs on my ubuntu 18.04
<mattip_>
I just needed to apt install lib*:i386 on some 32-bit system libraries
squeaky_pl has quit [Ping timeout: 268 seconds]
<antocuni>
uh? Why do you need i386 libraries on a 64bit build?
<mattip_>
I don't - this is a 32-bit build
<antocuni>
ah ok :)
<antocuni>
very nice
<mattip_>
I started with the 32-bit docker image on the 32-bit buildslave, if it builds py3.6 cleanly I will move to the 64 bit
<mattip_>
there are some test failures, but I think I worked them out
<antocuni>
mattip_: if you want to be very advanced, you could try to run the docker based tests and/or translations on azure pipelines
<antocuni>
(which might involve setting up a github mirror just to trigger the build :( )
<mattip_>
how much time can we take? A single own untranslated run on pypy2 is about 90 minutes
<antocuni>
you mean own tests?
<mattip_>
on py3.6 the same tests take 4 times as long
<mattip_>
yes
<antocuni>
we could split them into multiple jobs, one for each directory/group of directories
<mattip_>
translation is less than an hour, so that might be OK
<antocuni>
the main limit for azure pipelines is the amount of jobs you can run in parallel, but it's like 10 for free projects
<antocuni>
and, in my very limited experience with using them for hpy, they seem to run on FAST hardware
<antocuni>
the feeling is that jobs complete much faster than e.g. on travis
<mattip_>
we would have to redo the error summary reporting
<antocuni>
true, but note that they have builtin support for pytest; e.g. look here:https://dev.azure.com/pyhandle/hpy/_build/results?buildId=25&view=ms.vss-test-web.build-test-results-tab
<antocuni>
(you need to pip install pytest-azurepipelines)
<antocuni>
also, in my idea if we migrate to a workflow in which we can easily run tests for all/most platforms automatically on each PR, we should really arrive to the point in which we can aim to have always green builds
<antocuni>
which means less needs for the current summary reporting
squeaky_ has quit [Read error: Connection reset by peer]
squeaky_ has joined #pypy
<antocuni>
mattip_: about the speed of azure pipelines: the "pytest" stage of hpy takes ~12s on azure, and 34 seconds on my machine
<antocuni>
the whole job (including checkout, installing dependencies, etc) took 30s, which means that if I want to look at test results, it might be faster to execute "git push" than "py.test" :)
forgottenone has quit [Quit: Konversation terminated!]
<mattip_>
heh
<mattip_>
it doesn't show the last 5 test runs but it does have "new" for test failures
<antocuni>
yes, and it seems there is also some logic to autodetect flaky tests
squeaky_ has quit [Quit: Leaving]
<mattip_>
it does have "failing since" too
<antocuni>
yes
<antocuni>
I see lots of nice features which will make our life easier and ultimately lead to a better sw
<antocuni>
the biggest problem for us is that there does not seem to be an easy way to integrate with bitbucket/mercurial
<mattip_>
hggit works, it would be easy to set a cron job to push to a read-only repo on github.com
<antocuni>
but I don't think it's worth trying to do it manually since we are moving away from bitbucket anyway
<antocuni>
mattip_: yes exactly, I was thinking of something like that
<mattip_>
and add a "for more info/issues/PRs go here" readme
<antocuni>
when we migrate to gitlab it will make sense to try to have a better/deeper integration, but for now the cronjob might be enough
<mattip_>
on the other hand, I like it that we can ssh in to the build machine and directly run tests
<mattip_>
everything is set up and running, since the buildslaves need a proper environment
<mattip_>
if we move away from that, the various machines will suffer bitrot (dependencies, tools, ...)
<antocuni>
that's why I think it is very important to have a docker-based CI
<antocuni>
so that if needed, you can run them in the same environment
jvesely has quit [Quit: jvesely]
forgottenone has joined #pypy
rubdos has joined #pypy
CrazyPython has joined #pypy
CrazyPython has quit [Read error: Connection reset by peer]