cfbolz changed the topic of #pypy to: PyPy, the flexible snake (IRC logs: https://botbot.me/freenode/pypy/ ) | use cffi for calling C | the secret reason for us trying to get PyPy users: to test the JIT well enough that we're somewhat confident about it
<simpson>
Oh, wow, I'm so sorry, I totally misread your beginning question. I read `#define` instead of `#pragma`. Crap. I think that there's CFFI docs for this.
<amz3`>
is it possible to define extra functions in ffi.set_source?
<simpson>
You can put whatever you want in .set_source() and it'll go to the C compiler. What happens with your .cdef()? Does cffi vomit on the #pragma?
<simpson>
My hope would be that cffi ignores the #pragma in .cdef(), but the C compiler sees and uses it, and that all works together. I'm not sure though.
<amz3`>
no
<amz3`>
both ignore it
<amz3`>
but when I run the program, there is an error that says cdef is wrong and that the compiler reported another size for the struct
<simpson>
Hmm. I'm not sure I can help more. The cffi authors are usually on during European hours though.
<kenaan>
cfbolz py3.6 ef3c287b0add /pypy/interpreter/: don't crash if the locals aren't a dict. fix another corner case about annotations. (aside: this shows that removin...
asmeurer_ has quit [Quit: asmeurer_]
<mattip>
is there a design document for the changes in unicode-utf8? The merge with py3.5 is not simple,
<mattip>
many of the function interfaces have changed in handling unicode and there are functions in py3.5 that do not exist in default
<mattip>
in general, does py3.5 handle surrogates differently than default?
<cfbolz>
fijal: ^^
<cfbolz>
questions for you, I think
<mattip>
maybe I should try to arrange to meet with him virtually and share a screen to work through the harder parts of the merge
<cfbolz>
the other question I kind of have is what the status of py3.6 is. I've been implementing some tiny things there, but I have no feeling at all about the bigger picture
user24 has joined #pypy
amz3` has quit [Ping timeout: 276 seconds]
<mattip>
cfbolz: it would seem to me that not building the blake bindings via cffi is a major blocker
<Rotonen>
you first gotta brew install pyenv, though
<lesshaste>
Rotonen, oh I see. you downloaded the binary for pypy 6?
<Rotonen>
and start a new shell session, homebrew somehow injects pyenv into your shell session for you - not fully sure how that works
<Rotonen>
and there's a caveat that if you install any pyenv plugins, like pyenv-virtualenv (which is super useful IMO), those you'll have to inject into your shell session yourself (bashrc, bash_profile, they all come with on-terminal instructions when you install them, though)
<Rotonen>
if you just want to run it, that should work for you too, but i prefer to use pyenv for various reasons
<Rotonen>
(i also have funky requirements like testing stuff with both ucs2 and ucs4 on cpython 2.7, so i'm probably not the best yardstick for everything)
lazka has joined #pypy
<Rotonen>
or you explicitly want to translate your own pypy from source?
<mattip>
lesshaste: note that download is high-sierra, not sierra, apparently there is a difference
tbodt has joined #pypy
dddddd has joined #pypy
<lesshaste>
mattip, I think I have high sierra
<mattip>
+1
<Rotonen>
mattip: ah yes, is there someone currently tackling compiling backwards-compatibly on the latest one? i can still provide the lists of syscalls per version which need to be avoided
<lesshaste>
mattip, I was going to run the scipy tests, unless you have already done that
<lesshaste>
mattip, and try matplotlib on pypy 6
lazka has quit [Quit: Leaving]
<mattip>
lesshaste: it might be a bit frustrating. SciPy is not an end to itself, but a very large colleciton of tools to enable other stuff. You might be better off trying the other stuff directly
<lesshaste>
mattip, sorry which other stuff?
<mattip>
and coming back to scipy tests when something crashes or is impossibly slow
<mattip>
whatever you want to use SciPy for
<lesshaste>
mattip, oh I see.. I was only going to do it to be useful.. in case we don't have a full list of what works currently and what doesn't
<lesshaste>
(sw_vers tells you the OS version is seems)
necaris has joined #pypy
<lesshaste>
mattip, I like lists that one can tick off :)
<mattip>
lesshaste: don't let me stop you of course, please put the list somewhere public
<mattip>
Rotonen: we really need someone to tackle MacOS support. There are some open issues and pull requests, maybe if you could take a look and see if anything is relevant
<mattip>
for instance, do we need two binaries or can we do something at runtime?
* mattip
off, back in a bit
<Rotonen>
'two binaries'?
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<mattip>
one for sierra, one for high-sierra
asmeurer_ has joined #pypy
<Rotonen>
no, you can compile on high sierra for anything older, you just need to know which syscalls are introduced when, it's a bit ugly - for cpython i run configure and then patch pyconfig.h before compilation
necaris has quit [Ping timeout: 250 seconds]
<Rotonen>
the easiest path is to compile on an older os version as things are forwards compatible, that's what cpython is doing
<mattip>
do you lose anything by not supporting high-sierra syscalls? There is some clock thing or other that is missing
<Rotonen>
but then you're stuck with older unpatched compilers, of course
<Rotonen>
depending on how far back you want to reach, getentropy() is something you might miss
tbodt has joined #pypy
<Rotonen>
mattip: here's for example what i'm doing for cpython 3.6 for embedding that into an .app which is built on 10.13 and can safely be distributed to old versions up to 10.9: https://pastebin.com/cspRUNRi
<Rotonen>
but yeah, as a summary, no nanosecond resolution clocks (as if macs could do that anyway), no getentropy, some newer filesystem access stuff is not available
dfee2 has joined #pypy
necaris has joined #pypy
necaris has quit [Client Quit]
asmeurer_ has quit [Quit: asmeurer_]
<Rotonen>
and of course my mapping for 3.6 there is 1) not necessarily the full mapping despite me not having hit anything else yet 2) most likely different for python 2.7 3) even likelier different for pypy2.7
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
asmeurer__ has quit [Quit: asmeurer__]
Rhy0lite has quit [Quit: Leaving]
tbodt has joined #pypy
kanaka has quit [Ping timeout: 240 seconds]
tbodt has quit [Client Quit]
bomb has quit [Quit: VM gone wild]
asmeurer_ has joined #pypy
kanaka has joined #pypy
asmeurer_ has quit [Quit: asmeurer_]
tbodt has joined #pypy
dfee2 is now known as dfee
asmeurer has joined #pypy
asmeurer has quit [Quit: asmeurer]
lesshaste has joined #pypy
lesshaste has quit [Read error: Connection reset by peer]
asmeurer__ has joined #pypy
fryguybob has quit [Remote host closed the connection]
asmeurer__ has quit [Quit: asmeurer__]
marky1991 has joined #pypy
asmeurer_ has joined #pypy
marky1991 has quit [Read error: Connection reset by peer]
asmeurer_ has quit [Quit: asmeurer_]
fryguybob has joined #pypy
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
tbodt has joined #pypy
energizer has joined #pypy
energizer has quit [Remote host closed the connection]
energizer has joined #pypy
<dfee>
what kind of strategies are there to make pypy competitive with cpython for startup? is there a pre-compilation step pypy could do?
inhahe__ has joined #pypy
<cfbolz>
dfee: what exactly do you mean by "startup"?
<cfbolz>
the time till execution of the first line of user code?
<dfee>
cfbolz: yeah i think that's probably a good measure
<dfee>
well...
<cfbolz>
bit unclear. we have profiled what happens at PyPy's startup in a very long time (if ever)
<dfee>
pypy can achieve better performance than cpython as the JIT gets more usage data to better predict branch-paths (is that right?).
<cfbolz>
yes
__pv has quit [Remote host closed the connection]
<cfbolz>
dfee: that's a more nebulous concept called "warmup", ie how long does it take for pypy to achieve its maximum speed
<dfee>
so the first time through the code base, or maybe the first N times (for a low N?) pypy will be significantly slower than cpython.
<dfee>
cfbolz: yeah. i think that's what i'm referrring to.
<cfbolz>
ok
<dfee>
is there a notion of a pre-warmup stage?
<cfbolz>
yes, I suppose
<gsnedders>
there's also a cost to doing the tracing, which inevitably CPython doesn't have
<cfbolz>
yes
<dfee>
well, of course anything before warmup would be "pre-". but is there a way to introduce a pre-warmup phase between pre-pre-warmup and warmup? :)
<simpson>
dfee: You need to profile to see how PyPy actually behaves on your application. Pre-JIT interpreting is not necessarily slow, and warmups are not necessarily slow-then-fast.
<dfee>
simpson: that makes sense.
<simpson>
There was a great paper a few years ago which studied several JITs and concluded that 'JIT warmup' is a meme that isn't always real.
<gsnedders>
and during the warmup phase it's rarely a single jump from "slow" to "fast", but typically progressively getting faster, as fewer and fewer execution paths fall off the trace
<simpson>
And I think that one of the authors is here, but I can't remember the paper title or who worked onit.
<cfbolz>
Yep, always measure with the real workload before thinking too much about the problem
<dfee>
cfbolz: so i'm trying to backport a library. or maybe i won't, and will wait for pypy to catch up to 3.6. but anyway, there are actual code differences that need to be accounted for when going from cpython -> pypy.
<cfbolz>
dfee: which feature do you need?
<dfee>
so i haven't written in consideration of pypy, but i wonder if i should take that into account if i want to make a library that can run on pypy in the future
<simpson>
cfbolz: Yeah, that one. So good.
<dfee>
cfbolz: i've largely written using class-var annotations and f-strings
<cfbolz>
dfee: f strings work already
<dfee>
cfbolz: but i also rely on dict being ordered by default
<cfbolz>
Dicts are ordered
<cfbolz>
They have been since quite a while in pypy
<dfee>
cfbolz: really? not just collections.OrderedDict?
<dfee>
(again this is for a package i intend to publish on pypi)
<dfee>
so i'm largely looking at the tradeoff between supporting pypy and supporting mypy right now then...
<dfee>
and i already have mypy support
energizer has quit [Remote host closed the connection]
<dfee>
(or should i say, type annotation)? i haven't seen what Facebook offered this morning in terms of a type-checking package, so maybe there are other options besides mypy now.
energizer has joined #pypy
__pv has joined #pypy
<cfbolz>
dfee: yep, for those you need to wait
tbodt has quit [Quit: My Mac has gone to sleep. ZZZzzz…]