cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://quodlibet.duckdns.org/irc/pypy/latest.log.html#irc-end ) | use cffi for calling C | if a pep adds a mere 25-30 [C-API] functions or so, it's a drop in the ocean (cough) - Armin
jcea has quit [Ping timeout: 276 seconds]
famubu has joined #pypy
<famubu> Thanks.
dmalcolm has quit [Ping timeout: 265 seconds]
famubu has quit [Ping timeout: 246 seconds]
famubu has joined #pypy
<cfbolz> mattip: sorry for the breakage :-(. I can fix stuff after lunch, hope that's not too late (otherwise let's revert the list change)
dmalcolm has joined #pypy
Eyess has joined #pypy
alcarith1 has joined #pypy
SunDwarf has quit [Quit: Disconnect by user.]
alcarithemad has quit [Ping timeout: 260 seconds]
Dejan has joined #pypy
Dejan has joined #pypy
Dejan has quit [Client Quit]
<arigato> simpson: re transparent forwarders removed by the GC: that's not explicitly supported by RPython, no. it could be relatively easy to add it just to try
WGH_ has quit [Ping timeout: 260 seconds]
WGH_ has joined #pypy
Dejan has joined #pypy
Dejan has joined #pypy
lritter has joined #pypy
famubu has quit [Ping timeout: 246 seconds]
Dejan_ has joined #pypy
jcea has joined #pypy
Dejan_ has quit [Client Quit]
Dejan_ has joined #pypy
Dejan_ has quit [Client Quit]
Gustavo6046_ has joined #pypy
Gustavo6046 has quit [Ping timeout: 260 seconds]
Dejan has quit [Ping timeout: 260 seconds]
Gustavo6046_ is now known as Gustavo6046
famubu has joined #pypy
famubu has quit [Quit: leaving]
proteusguy has joined #pypy
tos9 has joined #pypy
<cfbolz> arigato: it's not *completely* trivial, I think: if the object is used by a currently running jit trace, the JIT will not expect its type to change
<cfbolz> mattip: the list problems should be fixed, sorry again for the breakage
<simpson> Another complication is that I technically need *conditional* transparent forwarding; I have chains of promises/futures which might not be resolved, and I only want to shorten the chains after resolution. My current approach involves a `resolution()` helper called at strategic locations.
Dejan has joined #pypy
Dejan has quit [Changing host]
Dejan has joined #pypy
Dejan_ has joined #pypy
Dejan has quit [Ping timeout: 260 seconds]
tsaka__ has joined #pypy
tsaka__ has quit [Client Quit]
<mattip> cfbolz: no urgency, thanks for the fixes
Dejan_ has quit [Quit: Leaving]
Dejan has joined #pypy
Dejan has joined #pypy
Dejan has quit [Changing host]
jcea has quit [Ping timeout: 250 seconds]
shunning has joined #pypy
<shunning> Hi guys, I'm facing a problem on centos 7 with the latest pypy3 7.3.3. I need to compile pypy by myself to remove some limits. However, it turned out that the I only have openssl 1.0.2 on centos7, so I get this problem
<shunning> .../lib-python/3/hashlib.py:173: RuntimeWarning: The _hashlib module is not available, falling back to a much slower implementation (/work/global/sj634/cornell-brg/pypy-pymtl3/lib_pypy/_pypy_openssl.pypy37-pp73-x86_64-linux-gnu.so: undefined symbol: SSL_CTX_set_min_proto_version)
<shunning>   RuntimeWarning)
<shunning> Then, I was able to compile PyPy, and create a virtual environment using -m venv, but then pip stopped working possibly due to the openssl issue
<shunning> $ pip install pytest
<shunning>   RuntimeWarning)
<shunning> WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
<shunning> WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/pytest/
<shunning> WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/pytest/
<shunning> WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/pytest/
<shunning> WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/pytest/
<shunning> WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/pytest/
<shunning> The error message ends with Could not fetch URL https://pypi.org/simple/pytest/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pytest/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
<shunning> ERROR: Could not find a version that satisfies the requirement pytest (from versions: none)
<shunning> ERROR: No matching distribution found for pytest
<shunning> WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
<nimaje> please use a pastebin for something like that
shunning has quit [Client Quit]
lritter has quit [Ping timeout: 240 seconds]
shunning has joined #pypy
<shunning> maybe the question is, does pypy3 7.3.3 require openssl 1.1.0? For CPython it says it supports 1.0.2 or 1.1.0
jcea has joined #pypy
<mattip> shunning: at the end of translation, the last step should have run "cd lib_pypy && pypy pypy_tools/build_cffi_imports.py"
<mattip> shunning: which will create lib_pypy/_pypy_openssl.*.so
<mattip> shunning: which will provide _ssl
<mattip> this will build with the the openssl lib on your system (you did install the build dependencies, right?)
<mattip> but let's back up: what limits do you want to remove by compiling yourself?
<mattip> ahh, hang on, importing the _ssl cffi library failed with "undefined symbol: SSL_CTX_set_min_proto_version"
<shunning> no worries, we are from Cornell and we always use a modified pypy for various purposes lol
<mattip> so yes, it seems you must use 1.1.0 or higher
<shunning> on my own laptop the compilation is very smooth with ubuntu 18.04/20.04, it's just than the server has centos 7 and openssl 1.0.2 is the highest one that yum provides
<mattip> which is a good idea anyway, older openssl versions suffer from known security issues
<mattip> even better: rebuild the cffi imports with
<mattip> cd lib_pypy && <new-pypy> pypy_tool/build_cffi_imports.py --rebuild --embed-dependencies --only _ssl
<shunning> ok, let me try this!
<mattip> which will download openssl 1.1.1k and rebuild the _ssl module with it
<mattip> the "install_opensslsh" script will try to permanently install opensll to /usr/local, where the
<mattip> --embed-depenencies option will build openssl each time
<shunning> oh by the way i don't have sudo ....
<mattip> right, so the second solution is better for you
<shunning> but i can try embed dependencies
<mattip> while it is building, it would be interesting to hear what you are modifying, maybe we can learn something or provide a better experience
<shunning> nah we modify too much ....
<shunning> we are the group who runs pypy for hardware simulation in Python
<shunning> Carl helped us a lot in the past
<mattip> ahh, OK
<shunning> I can mention a few modification we had to do to achieve our purposes
<shunning> LIMIT_MAP_ATTRIBUTES = 1000000 (used to be 80)
<shunning> PENDINGFIELDSTRUCT = lltype.Struct('PendingField',
<shunning>                                    ('lldescr', OBJECTPTR),
<shunning>                                    ('num', rffi.SHORT),
<shunning>                                    ('fieldnum', rffi.SHORT),
<shunning>                                    ('itemindex', rffi.INT)) we have to enlarge these shorts to INT and modify the token calculation mechanisms
<shunning> if name1 == 'trace_limit' and int(value) > 2**14:
<shunning>                             raise TraceLimitTooHigh
<shunning> remove this trace limit, compile using bigmodel
<shunning> and the huge TLB flag
<shunning> flags = MAP_PRIVATE | MAP_ANONYMOUS | MAP_HUGETLB
<mattip> is this public in a repo somewhere?
<mattip> kind of hard to digest multiple lines on IRC
<shunning> yes
<shunning> it's really about when you try to JIT really really large loops with like a thousand different small functions
<shunning>  undefined symbol: OPENSSL_sk_num
<shunning> internet says 1
<shunning> It looks like in version 1.1.0f of openssl the symbol OPENSSL_sk_num has moved to libcrypto.a. The build of python 3 didn't seem to link that in, hence the missing symbol.
<shunning> thanks for the help mattip. I think I know how to solve them using cffi_import
shunning has quit [Quit: Connection closed]
<mattip> ok, although if you are using latest HEAD of default or py3.7, --embed-dependencies should get you 1.1.1k, which is the recommended version
<mattip> shunning: (for the logs) I opened an issue there about how you can update and use the package.py script to automate the OpenSSL build
<mattip> shunning: (for the logs) also, I wonder if the really really large loop problem is the same one cfbolz is working on in the open-ended-traces-3.7 branch
<cfbolz> mattip: their use case is really quite special
<mattip> ok
<cfbolz> (they have a hardware emulation language based on python. The traces they produce are really really long, but optimize super well)
<Dejan> they mention PyPy too there
<cfbolz> Dejan: nice find!
oberstet_ has quit [Quit: Leaving]
<mattip> it is a shame cppyy does not get more attention
holdsworth_ has joined #pypy
nulano has quit [Ping timeout: 265 seconds]
holdsworth has quit [Ping timeout: 265 seconds]
sknebel_ has joined #pypy
Techcable has quit [Ping timeout: 265 seconds]
energizer has quit [Ping timeout: 265 seconds]
nulano has joined #pypy
gsnedder1 has joined #pypy
gsnedders has quit [Ping timeout: 240 seconds]
sknebel has quit [Ping timeout: 240 seconds]
Techcable has joined #pypy
energizer has joined #pypy