cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
<_aegis_>
arigato: I have a small list of libraries the symbol might come from, including my own executable. on mac/linux it's sufficient to just dlopen(None)
<_aegis_>
a dlsym hook would be perfect for me
<_aegis_>
I'm loading hundreds of functions from several libraries and the "host" executable that embeds python
<_aegis_>
I'm using a single ffi/lib pair for this, and a script that ingests dozens of headers to precompile them (a CompiledFFI object)
<_aegis_>
it would be a massive pain to use several (5-10?) ffi/lib pairs instead of just the omnibus one
<_aegis_>
but as far as I can tell there's no way for CFFI to load from multiple libraries, and there's no way to say "load from my own executable symbol space" either
<_aegis_>
a wrapper library that re-exports the symbols on windows would work in theory but I don't know how to do that offhand, and it honestly feels way less clean to do that than to just use my own symbol loader
<_aegis_>
I've had other uses for a symbol loader in cffi (like my crossldso project), so if a dlsym hook is far enough removed from my crimes for you to consider it, I'm fine with that and it's still much nicer for me than faking the lib object
<_aegis_>
(this isn't a single library <-> python bindings project, it's a complex desktop application that exposes a bunch of artisinal APIs at once from different shared libs)
<_aegis_>
doing it this way basically allows me to create a new C header in the right place and start using that symbol immediately from python, which is extremely convenient
<antocuni>
it tries to find "openblas_lapack_info" (which should be installed by apt install libopenblas-dev), but the check fails because of "undefined reference to `zungqr_'" (line 289)
<antocuni>
so, if you wanted a reproducer for those bug, I might have found it :)
<antocuni>
and if you also have any insight of how to fix/workaround the problem, I'd be glad to hear
<antocuni>
(note that I get this problem only on my ubuntu-based builds. The manylinux builds work fine, where I install openblas by doing "yum install -y openblas-devel"
<mattip_>
ok, I can look a bit later
<antocuni>
thanks
<mattip_>
it is looking in /usr/lib, and finds an openblas, but it must be the wrong one?
<antocuni>
I think it is the ONLY one
<antocuni>
it's a plain ubuntu14.04 image with apt install libopenblas-dev
<antocuni>
FWIW, it doesn't seem to be a problem of pypy
<mattip_>
the system openblas is too old. NumPy downloads one using `python numpy/tools/openblas_support.py`
<antocuni>
I managed to get the same error with cpython3.4, by doing pip install numpy==1.14.3 && pip install scipy==1.2.2
<antocuni>
ah ok
<mattip_>
I think scipy's CI also downloads a more recent openblas
<antocuni>
is it enough to run "openblas_support.py"?
<mattip_>
that returns the location where it unpacks the openblas, then you need to set INCLUDE/LDFLAGS
<antocuni>
ah, but then it cannot work now that I think of it
<antocuni>
since I am building ubuntu wheels, I want to use the system libraries
<antocuni>
else the produced wheels will be unusable
<antocuni>
maybe it is enough to use a newer ubuntu image
<mattip_>
which ubuntu are you currently using?
<antocuni>
14.04
<antocuni>
(I mean, a docker image of ubuntu 14.04)
<mattip_>
yeah, that's dead. 16.04 is EOL in April 2020 I think
<antocuni>
or maybe I could just drop support for scipy: if someone wants a scipy wheel, it should switch to the manylinux ones :)
<mattip_>
that's weird, now it works, and a second ago it didn't
<antocuni>
ok
<antocuni>
I should update it to point to manylinux, though
<mattip_>
wow, that is worth a tweet/blogpost ! Thanks for doing that !
<antocuni>
yes, I want to polish things up though
<antocuni>
also, we should do a pypy release shipping a pip which automatically uses manylinux wheels, else people will be confused
<Dejan>
i wonder will it work on Fedora/CentOS
<antocuni>
they should
<mattip_>
right, HEAD does that, I should release it soon.
<mattip_>
I wanted to transition the linux buildbots to the docker-based portable one first
<antocuni>
mattip_: ah, something to fix is probably openblas: right now I am installing it using yum install, but I seem to understand that numpy/scipy does it differently
YannickJadoul has joined #pypy
<mattip_>
well, the real solution to that is to get those projects to build pypy manlinux2010 wheels
<YannickJadoul>
antocuni: Is it not possible to upload PyPy manylinux wheels to PyPI?
<antocuni>
YannickJadoul: yes it is, but only if we are the project maintainer
<YannickJadoul>
Ah, yes, ofc
<antocuni>
eventually I hope that numpy/scipy/etc will upload their own pypy wheels, but you should ask mattip_ about that :)
<mattip_>
it really is just one more build in their matrix now
oberstet has quit [Remote host closed the connection]
<YannickJadoul>
mattip_: Re. the wheel filenames: did you try this build on Windows?
<YannickJadoul>
Because on linux, the filenames are as expected ({package_name}-{package_version}-pp372-pypy3_72-manylinux2010_x86_64.whl)
oberstet has joined #pypy
<mattip_>
and on windows?
<YannickJadoul>
There I get {package_name}-{package_version}-pp372-pp372-win32.whl
<YannickJadoul>
(But only for Python 3, for Python 2, I get {package_name}-{package_version}-pp272-pypy_41-win32.whl)
<mattip_>
that is an old versions of all linux, windows, python2, python3. What do you get on HEAD? You can try a nightly build