<arigato>
ah no, indeed, but the difference is only for i==2
antocuni_ has joined #pypy
<arigato>
that means that just counting is wrong, again for low-level CPU cache reasons
<arigato>
the difference might seem big, but in reality, this difference is only writing "1" in every other entry of the list in order
flechebleue has joined #pypy
<arigato>
which is much, much faster than writing "1" in less predictible locations
<lesshaste>
I get 192570204 for sieve_var and 242570202 for sieve for with 10**8 for example
<lesshaste>
arigato, so what is the time dominated by if not that line?
<arigato>
yes, and the difference is 10**8 / 2
<arigato>
I think the time is dominated by that line but only when it is executing for non-very-small values of i
<arigato>
i.e. when it is writing "1" at what looks like random places to the CPU
antocuni has quit [Ping timeout: 276 seconds]
yuyichao_ has quit [Ping timeout: 276 seconds]
<lesshaste>
arigato, so is the current hypothesis that the extra iterations sieve_var does are just very cheap?
<lesshaste>
arigato, and the remaining ones that both sieve and sieve_var do are the expensive ones for some reason?
<arigato>
yes
<lesshaste>
sorry.. the extra iterations sieve does
<njs>
it basically does 1 extra iteration, and that's a straight bulk pass through memory; later on the memory accesses are way more unpredictable and probably taking cache misses all the time
<lesshaste>
njs, it does many more than 1 iteration
<lesshaste>
1 extra iteration I mean
<arigato>
it does what njs said, plus what I said in my SO answer
<arigato>
both of which are relatively very cheap
<lesshaste>
njs, I am trying 3*10**8 where sieve does 745994568 inner iterations
<njs>
lesshaste: all the other iterations bounce out on the if nums[i] == 0 part
<lesshaste>
and sieve_var does 595994570
<lesshaste>
njs, that is the count of the innermost loop
<njs>
lesshaste: the first iteration sets nums[i] = 1 for all the even i, so after that it's very very similar to doing a range with a step of 2
<cfbolz>
how do caches even
<lesshaste>
arigato, and windows just remains a mystery? It's sieve_var is definitely slower in windows than sieve
<arigato>
lesshaste: I tried to give in my answer "it's 2x faster if you do that instead", i.e. I don't know why you get 10% differences but also don't think we should worry a lot
<lesshaste>
it does seem odd that the extra 149999998 iterations apparently cost nothing
<lesshaste>
but I will leave it at that :)
<njs>
lesshaste: yes, those inner iterations are all on the very first pass, and they're a super-predictable linear memory scan, so the prefetcher probably kicks in and you get full memory bandwidth instead of memory latency
<lesshaste>
njs, interesting
<cfbolz>
arigato: you can get a 7x speedup by switching to bytearray ;-)
<arigato>
of course it doesn't make sense that doing more work runs in 10% less time, but that's within the nonsense I'm getting used to for CPUs
<arigato>
cfbolz: ah, bytearrays :-) of course
<lesshaste>
cfbolz, nice!
<njs>
lesshaste: have you run it repeatedly in a loop to make sure those numbers have stabilized after warmup etc etc, at least as much as they're going to?
rokujyouhitoma has quit [Ping timeout: 246 seconds]
amaury has joined #pypy
jcea has joined #pypy
ronan has quit [Ping timeout: 240 seconds]
horrid has joined #pypy
ronan has joined #pypy
<horrid>
Hello, I have managed to make wxpython-cffi work with PyPy5.8.0 and matplotlib and show plots on the screen (steps are here: https://gist.github.com/HorridBro/8b650dcedd4f75399fa94b8e00f23e34). It seems that there are many modifications needed for this to work and I am not sure how it's best to continue to push this code on the 2 repos, especially
<horrid>
in wxpython-cffi. Can you please give me some advice?
ELFrederich has joined #pypy
ELFrederich has quit [Client Quit]
ELFrederich has joined #pypy
antocuni_ has joined #pypy
and1 has joined #pypy
<and1>
Hi, I've upgraded from PyPy 5.4.1 to 5.8.0 and I'm getting "TypeError('ref() does not take keyword arguments"
<and1>
it's in cassandra-driver package
antocuni_ is now known as antocuni
<antocuni>
and1: could you please paste the full traceback?
amaury has quit [Ping timeout: 240 seconds]
<and1>
It's problematic, because it seems it's an inner exception: NoHostAvailable: ('Unable to connect to any servers', {'127.0.0.1': TypeError('ref() does not take keyword arguments',)})
<antocuni>
the only "ref()" function I can think of is weakref.ref; but without more info, it's a bit problematic to help
<antocuni>
ok, but this is probably by accident. I claim that the cassandra code should not rely on the fact that "callback" is a keyword, since on CPython it is not
yuyichao_ has quit [Read error: Connection reset by peer]
<antocuni>
I bet that this precise line of code is executed only on PyPy but never on CPython?
yuyichao_ has joined #pypy
<and1>
I checked it on CPython 2.7.6 and it works also
<antocuni>
earlier, it simply ignored callback silently
<and1>
yes, but what about compatibility?
<antocuni>
it was a bug, we fixed it
<antocuni>
you should report the issue to cassandra
<mattip>
horrid: cool! In your script I only see one change to wxpython-cffi, in line 28. Am I missing something?
<mattip>
horrid: anyhow, the path forward would probably be a pull request to wxpython-cffi,
<mattip>
with both the fix and documenting your steps (use of wxWidgets 3.0.3 seems critical and not mentioned on the README)
<and1>
antocuni: ok
cstratak has quit [Quit: Leaving]
<antocuni>
pypy's c6167130f731 is the changeset which disabled the callback keyword argument
<horrid>
mattip: yes,that is the only modified line, but the problem is more deep. I saw that core.pi is generated faulty. In a normal build.py you shouldn't be needing to remove parts of a generated .pi. This is one of my dillemas that I haven't figured out
<antocuni>
we did it in the stdlib-2.7.12 branch, because now the stdlib contains a test which explicitly checks that ref(..., callback=...) fails
<horrid>
mattip: I'll proceed with a pull request, but don't know exactly where to start working on fixing the faulty .pi generation
and1 has quit [Quit: Leaving]
<horrid>
mattip: do you have any pointers from where I can grab this to analyze?
<mattip>
horrid: there is some documentation on the original GSOC work somewhere, let me look it up
cstratak has joined #pypy
<ELFrederich>
I have an argument to a function "const char dataset_name[WSO_name_size_c + 1]"... and above I put "#define WSO_name_size_c ..." and when I try to build I get this... cffi.error.FFIError: :9: unsupported expression: expected a simple numeric constant
commandoline_ has joined #pypy
<mattip>
horrid: you may find this blog helpful http://waedt.blogspot.com, reading from oldest to newest
<mattip>
ELFrederich: that is in ffi.cdef() correct? AFAIK pycparser does not understand macros like 'WSO_name_size_c +1'
<ELFrederich>
mattip: I get same error without the "+ 1"
<horrid>
mattip: Thank you ! I will start reading and come back to you .
<ELFrederich>
mattip: if I go and change #define WSO_name_size_c ..." to "#define WSO_name_size_c 128" then it works... even with the +1
<mattip>
horrid: I may have been too hasty, looking over that he does not go into enough detail
<mattip>
ELFrederich: well, as usual when I try to help with CFFI I am wrong, so I will just be silent
<ELFrederich>
;-)
<mattip>
horrid: you may want to file an issue on the bitbucket repo of wxpython-cffi, amaury also rarely shows up here
<ELFrederich>
I have a feeling this stuff used to work... I have a file in Git from 2015 (the last time I tried porting my Cython stuff to cffi) that has this kind of stuff in it
bjs has quit [Ping timeout: 255 seconds]
flechebleue has quit [Ping timeout: 240 seconds]
<mattip>
horrid: I found this link on the ProjectPhoenix web site, called 'HowTheBuildToolsWork',
<ELFrederich>
any cffi people here?... know how I can have a function in a cdef which takes a character array deifned like... "const char dataset_name[WSO_name_size_c + 1]"? ... maybe fijal?
<LarstiQ>
const char *?
Fleche has joined #pypy
<ELFrederich>
LarstiQ: that might work for function arguments but what about members of structs?
<LarstiQ>
ELFrederich: I'd have to direct you to someone with cffi experience
<LarstiQ>
and/or the documentation
* LarstiQ
doesn't know atm
commandoline_ has quit [Ping timeout: 240 seconds]
commandoline_ has joined #pypy
Ryanar has joined #pypy
DanielWHolth has joined #pypy
yuyichao_ has quit [Read error: Connection reset by peer]
commandoline has quit [Quit: Bye!]
commandoline has joined #pypy
<mattip>
ELFrederich: for people who might join later, what version of cffi?
<ELFrederich>
mattip: I'm on the latest that pypi gave me... 1.10.0 looks like
<mattip>
ok
<cfbolz>
ELFrederich: could you post a self contained example?
marky1991 has joined #pypy
<mattip>
ELFrederich: couldn't resist. For me it works if I do
yuyichao_ has quit [Remote host closed the connection]
yuyichao_ has joined #pypy
amaury has joined #pypy
_main_0 has quit [Read error: Connection reset by peer]
__main__ has joined #pypy
__main__ has quit [Read error: Connection reset by peer]
__main__ has joined #pypy
stevie has quit [Ping timeout: 260 seconds]
arigato has joined #pypy
<ELFrederich>
sup arigato ... know how I can have a function in a cdef which takes a character array defined like... "const char dataset_name[WSO_name_size_c + 1]"?
jcea has joined #pypy
<arigato>
if that's a parameter, then you can say "const char dataset_name[]" instead; these two are 100% equivalent C code
<ELFrederich>
arigato: okay... thanks. And for struct members I can put "const char dataset_name[...]" right?
<arigato>
yes
* ELFrederich
forgot that they're equivalent.... though if I remember correctly something like "int data[height][width]" needs one of those
<arigato>
yes
<arigato>
then it becomes more complicated
lesshaste has quit [Ping timeout: 260 seconds]
<arigato>
I'd have to check again, but I think you can say "int data[height][widht]" if height and width are not expressions but just constants
lesshaste has joined #pypy
lesshaste is now known as Guest81854
Guest81854 has quit [Client Quit]
<ELFrederich>
A suggestion please. I have a C library where I have a corresponding .py file for each .h file (120 files) where I basically mark up the function definitions with extra meta-data about whether things are input or output. So for <foo/bar.h> I have a definitions/foo/bar.py which I use to generate a proper foo/bar.py and a foo/bar_build.py
<ELFrederich>
... so my question. With 120 *_build.py files where they all share a lot of structures, do I need to cdef all those typedefs and struct definitions in each one?
yuyichao_ has quit [Remote host closed the connection]
yuyichao_ has joined #pypy
<ELFrederich>
... or is there a way for one FFI to "import" a cdef from another FFI object?
<ELFrederich>
I realize that C works where each compilation unit basically #includes something with definitions so maybe repeating "#define unsigned int tag_t" on all 120 files isn't terrible. Was just wondering if there is a better way