stebalien changed the topic of #ipfs to: Heads Up: To talk, you need to register your nick! Announcements: go-ipfs 0.4.19 and js-ipfs 0.34 are out! Get them from dist.ipfs.io and npm respectively! | Also: #libp2p #ipfs-cluster #filecoin #ipfs-dev | IPFS, the InterPlanetary FileSystem: https://github.com/ipfs/ipfs | Logs: https://view.matrix.org/room/!yhqiEdqNjyPbxtUjzm:matrix.org/ | Forums: https://discuss.ipfs.io | Code of
<deltab>
dynodetk[m]: or change the domain to m.twitter.com
griffinbyatt has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
maxzor has quit [Remote host closed the connection]
<dynodetk[m]>
that will still take you to the newer layout if you have a newer agent
<dynodetk[m]>
which is arguably still better for viewing longer threads, but honestly it's seemed to me to be even laggier than the desktop version
griffinbyatt has joined #ipfs
griffinbyatt has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
griffinbyatt has joined #ipfs
griffinbyatt has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
griffinbyatt has joined #ipfs
opal has quit [Remote host closed the connection]
opal has joined #ipfs
rcat has quit [Quit: Lost terminal]
dsiypl4_ has quit [Ping timeout: 248 seconds]
nvmq has quit [Quit: Leaving]
griffinbyatt has quit [Client Quit]
phs^ has quit []
rubio has joined #ipfs
tsrt^ has joined #ipfs
<rubio>
Hi! I'm planning to install a productive ipfs node on my datacenter. I want to help the network and maybe replace my statics hosts for this new technology.
<rubio>
I have a question... If I upload a web (some files), what happens when I upload the new version? I was not able to find an article on cleaning, removing old files (mark file as obsolete and the when there are no more access to this file for a ttl time remove it from all the nodes on the world).
el_rubio has joined #ipfs
rubio has quit [Read error: Connection reset by peer]
el_rubio has quit [Read error: Connection reset by peer]
el_rubio has joined #ipfs
mowcat has quit [Remote host closed the connection]
el_rubio has quit [Read error: Connection reset by peer]
el_rubio has joined #ipfs
jakehemmerle has joined #ipfs
user_51_ has joined #ipfs
verin0x has quit [Ping timeout: 246 seconds]
verin0x has joined #ipfs
user_51 has quit [Ping timeout: 248 seconds]
el_rubio has quit [Read error: Connection reset by peer]
el_rubio has joined #ipfs
el_rubio has quit [Read error: Connection reset by peer]
el_rubio has joined #ipfs
<fiatjaf>
el_rubio, there's no way to do ttls
<fiatjaf>
el_rubio, there's no way to keep track of what you have pinned or anything like that
spinza has quit [Quit: Coyote finally caught up with me...]
vandemar has quit [Ping timeout: 255 seconds]
vandemar has joined #ipfs
toxync01 has quit [Read error: Connection reset by peer]
toxync01 has joined #ipfs
spinza has joined #ipfs
mack[m] has joined #ipfs
Nd-969-M has joined #ipfs
fredthomsen has joined #ipfs
vandemar has quit [Ping timeout: 248 seconds]
vandemar has joined #ipfs
dsiypl4_ has joined #ipfs
fredthomsen has quit [Quit: Leaving.]
ilyaigpetrov has joined #ipfs
tombusby has joined #ipfs
<ilyaigpetrov>
Hi. I want to serve IPFS/IPNS data from a custom domain. I've tested DNSLink and I miss the ability to provide HTTP-redirects, e.g. I want https://example.com/old/path.ext to serve a redirect ot https://example.com/new/path.ext. It may be a relative link (e.g., `../../foo/bar.ext`) or absolute (e.g., `/some/thing.ext`) and the root is defined in the DNSLink TXT record (e.g., dnslink=/ipfs/<hash>/path/to/the/root). I want to
<ilyaigpetrov>
know what would be the best way to approach this problem, e.g., would it be better to dive into IPLD or maybe IPFS already has something like it. Any advice is welcome, thanks.
dsiypl4__ has joined #ipfs
chris613 has quit [Ping timeout: 258 seconds]
<ilyaigpetrov>
In a practical solution redirect link may be an arbitrary URL, even leading to another domain name.
chris613 has joined #ipfs
dsiypl4__ has quit [Client Quit]
dsiypl4_ has quit [Ping timeout: 252 seconds]
chris613 has quit [Ping timeout: 246 seconds]
fredthomsen has joined #ipfs
cwahlers has quit [Ping timeout: 246 seconds]
clemo has quit [Remote host closed the connection]
clemo has joined #ipfs
shibesh[m] is now known as shibesh1337[m]
shibesh1337[m] is now known as shibesh[m]
sammacbeth has quit [Quit: Ping timeout (120 seconds)]
sammacbeth has joined #ipfs
themsay has quit [Ping timeout: 248 seconds]
MuffinPimp has quit [Read error: Connection reset by peer]
zeden has joined #ipfs
MuffinPimp has joined #ipfs
sammacbeth has quit [Ping timeout: 248 seconds]
fredthomsen has quit [Quit: Leaving.]
sammacbeth has joined #ipfs
<ilyaigpetrov>
ok, my current idea is to run my own reverse proxy in front of the standard ipfs http gateway. If a text file served from the gateway has content like "IPNS_REDIRECT 307 /some/path/file.ext" then proxy replies to a user with a redirect.
<DokterBob>
`handleAddProvider` messages don't appear in logs after some time
Nd-969-M has quit [Ping timeout: 258 seconds]
reit has quit [Quit: Leaving]
chris613 has joined #ipfs
<SchrodingersScat>
ilyaigpetrov: I've reversed proxied with apache2 and nginx, not sure whan you're redirecting to but most should probably be able to do that. Although I'm not sure I completely understand your question.
<ilyaigpetrov>
SchrodingersScat: I want to have a http gateway, but for some files/paths I want to be able to set (http) redirects to newer paths
<SchrodingersScat>
newer ipfs/ipns links?
<ilyaigpetrov>
ipns, ipfs is immutable
<ilyaigpetrov>
IPLD has a concept of a link, maybe I can replace /ipns/example.com/foo/bar.ext with a IPLD-link pointing to some other file and handle this IPLD-links specifically in my reverse proxy
<ilyaigpetrov>
as a poor man option a may replace bar.ext with a text file with special content like `IPNS_REDIRECT 307 ../bla/bla` and handle this content in my reverse proxy specifically
<ilyaigpetrov>
And being more specific we are talking about DNSLinks which I guess is a subset of /ipns/ used with domain names.
<SchrodingersScat>
Am I way off base or even in the same sport? for example I have 'location /ipns/ { proxy_pass http://127.0.0.1:8080/ipns/; }' as part of my config. so are you wanting something like that + 'location /ipns/old.tld/examples { proxy_pass http://127.0.0.1:8080/ipns/new.tld/newthing; }'
pecastro_ has joined #ipfs
<ilyaigpetrov>
no, I'm not moving from one domain to another. I may have some popular file, e.g., ilyaigpetrov.me/popular.file, and one day I may want to redirect this file to another path for SEO purposes, e.g., ilyaigpetrov.me/cool-verbose-new-path/file.ext, so I need serving a http redirect for the old path.
pecastro has quit [Ping timeout: 250 seconds]
<ilyaigpetrov>
SchrodingersScat: as the default http ipfs gateway can't serve redirects then I have to use a reverse proxy. And stuffing every new redirect into nginx config is not a solution, I want redirects to be preserved inside IPNS maybe which may be controlled by my client (so if I provide any kind of such static files hosting over ipfs to someone).
sammacbeth has quit [Quit: Ping timeout (120 seconds)]
sammacbeth has joined #ipfs
mrsBeasley has joined #ipfs
sammacbeth has quit [Quit: Ping timeout (120 seconds)]
sammacbeth has joined #ipfs
mazeto has joined #ipfs
mrsBeasley has quit []
vandemar has quit [Ping timeout: 252 seconds]
vandemar has joined #ipfs
m_tadeu_ has joined #ipfs
jakehemmerle has joined #ipfs
m_tadeu has quit [Ping timeout: 252 seconds]
gde33 has quit [Ping timeout: 252 seconds]
gde33 has joined #ipfs
sammacbeth has quit [Ping timeout: 250 seconds]
reit has joined #ipfs
sammacbeth has joined #ipfs
blallo has quit [Ping timeout: 245 seconds]
cyfex has quit [Ping timeout: 248 seconds]
blallo has joined #ipfs
cyfex has joined #ipfs
Adbray has quit [Remote host closed the connection]
pvh_ has joined #ipfs
nt80 has joined #ipfs
sammacbeth has quit [Ping timeout: 258 seconds]
sammacbeth has joined #ipfs
seba- has joined #ipfs
DavidPH has joined #ipfs
jakehemmerle has quit [Ping timeout: 268 seconds]
jimmied has quit [Ping timeout: 252 seconds]
<swedneck_test>
is the wikipedia copy on ipfs still being updated?
<rubio>
Hi again, an other question: I have a use case where I need to know the address of the data before storing the data itself. I have seen IPNS can help with this by creating a new key and getting the hash and later changing the target. But then IPNS has a ttl/lifetime of about 24hours or whatever I setup. Now what if I want my content to be there for ever? O will need to republish the IPNS every N hours for eternity?
t4skforce has joined #ipfs
<DarkDrgn2k[m]>
IPNS lives forever.
<DarkDrgn2k[m]>
as long as some one is there to answer for it
<DarkDrgn2k[m]>
but IPNS is VERRRRRRYYYYYY slow due to all the natting thats happening on the ipfs network
<DarkDrgn2k[m]>
i think the TTL is more like a DNS TTL where it lives only X on some one elses server before it needs to be refreshed
<DarkDrgn2k[m]>
when doing our IPFS Live Video Stream, we had to publish the hashes outside of IPFS becaseu it was to slow. IPNS PubSub did help with the speed but still was not predictable enough for our needs
Adbray has joined #ipfs
<rubio>
So if a like a create my new startup over ipfs I should use dnslink and not ipns? IPFS is ready for productive use but IPNS is not, right?
<DarkDrgn2k[m]>
honestly i dont think ipfs or ipns is ready for production.. but thats my huble opionon :P
<DarkDrgn2k[m]>
but ipns resolves are VERY slow
<DarkDrgn2k[m]>
at least initially
<aschmahmann[m]>
rubio: DarkDrgn2k You do actually have to republish your IPNS key repeatedly. The reason for this is that nodes in the DHT will eventually disconnect from the network, dump your data, etc. and you will need to republish it. Unlike with IPFS, IPNS does not currently allow other nodes to republish the latest entries on your behalf.
<aschmahmann[m]>
Additionally, while the NATing is a problem for DHTs/open distributed systems in general that's not the primary reason IPNS is slow compared to IPFS. The primary cause for IPNS being slower than IPFS is that while IPFS only needs a single response from the DHT that says "here are some peers that might have your data" IPNS waits for many (something like 16) peers to say "here is the data". IPNS does this in order to make
<aschmahmann[m]>
it likely that you have the **latest** data, and not an older version of it.
shibesh[m] has left #ipfs ["User left"]
<DarkDrgn2k[m]>
aschmahmann: my info may be a year our of date but when speaking with ipfs devs that where at the ourNetworks confrence, they blamed it primarily on NAT
<aschmahmann[m]>
DarkDrgn2k: My understanding here is that NATs cause bad DHT performance. Logically this should affect both IPFS and IPNS since they both leverage the DHT. However, IPNS is hust much more than IPFS because it needs much more out of the DHT than IPFS does (e.g. 16 responses instead of 1). Seem reasonable?
Jesin has joined #ipfs
<DarkDrgn2k[m]>
yeh i guess it does make sens that IPNS would hit more nated nodes then IPFS
<rubio>
We are about to lunch a project like https://storycorps.org in Argentina and I think IPFS should be the best storage for this as the project is for saving interviews for ever and ensure that they are no banned at all.
<swedneck_test>
i'd say IPFS is production ready if you want to do very simple stuff
<DarkDrgn2k[m]>
ipfs eats SOOO MUCH resources when "idling" :/
<swedneck_test>
like using IPFS for websites? sure
<rubio>
Using dnslink will still depends on one cluster of servers to hold the links permanently...
<DarkDrgn2k[m]>
notice the bandwith requirements on most days
<aschmahmann[m]>
DarkDrgn2k: What about IPNS over PubSub was causing you problems with your livestreaming? Were you using floodsub or gossipsub?
<DarkDrgn2k[m]>
uhh tehre is a differnce?
<DarkDrgn2k[m]>
im trying to bring my samle stream up
<DarkDrgn2k[m]>
but i think the pen drive is fried hang up
<aschmahmann[m]>
gossipsub should be much less resource intense and still provide fairly stable message routes. Right now it's only available in go-ipfs, but the js-ipfs implementation is being worked on.
<DarkDrgn2k[m]>
this was all go-ipfs
<DarkDrgn2k[m]>
when we buiolt the stream js was not avaialble at teh time
<aschmahmann[m]>
there's a config option to use gossipsub instead of floodsub. If you're not using it I'd try it out and see if it's better for your use case.
<aschmahmann[m]>
It'll still have slow initial resolution time, but the updates should be fairly quick
<DarkDrgn2k[m]>
i thikn the issue was they would"stlal out"
<DarkDrgn2k[m]>
and not update fast enough
mowcat has quit [Remote host closed the connection]
sammacbeth has quit [Quit: Ping timeout (120 seconds)]
sammacbeth has joined #ipfs
<DarkDrgn2k[m]>
cant seem to resolve my ipns today
Sentark has quit [Ping timeout: 245 seconds]
<aschmahmann[m]>
:(
Jesin has quit [Quit: Leaving]
<aschmahmann[m]>
I'm currently working on some IPNS over PubSub improvements so any issues you're having it'd be great to hear about.
ilyaigpetrov has quit [Quit: Connection closed for inactivity]
cwahlers has joined #ipfs
<aschmahmann[m]>
The empty router currently refers to floodsub, if you put "gossipsub" as the Router you'll be using gossipsub. If you're interested more in gossipsub I'm happy to explain more, but I'd also recommend reading https://github.com/libp2p/specs/tree/master/pubsub/gossipsub
<DarkDrgn2k[m]>
if gossipsub compatible with ipfs.io atm
<DarkDrgn2k[m]>
hopefully it will be better then before !
el_rubio has joined #ipfs
<DarkDrgn2k[m]>
seems QmVNahAoBPjunHnR1uHQnZQtCxQ9aciyTSoJUk1KZ6Z2Kc rsolves but its just as slow next time
maxzor has joined #ipfs
<DarkDrgn2k[m]>
direct to the node works alright (if anyone has cjdns http://[fc58:a389:4e0f:bdee:21e1:ff06:bb23:167e]/video-player/)
rubio has quit [Ping timeout: 258 seconds]
<aschmahmann[m]>
To me that probably indicates that the pubsub peers haven't actually found each other. The easiest way to see this would be to spin up two IPFS nodes and see the difference between regular resolving and resolving after you've done an "ipfs swarm connect [otherPeer's multiaddr discovered from ipfs id]"
<DarkDrgn2k[m]>
aschmahmann: while i have you here got a question for you
el_rubio has quit [Read error: Connection reset by peer]
<DarkDrgn2k[m]>
is there any way to get the ID of a remote peer without exposing API?
el_rubio has joined #ipfs
<aschmahmann[m]>
I'm not sure what you mean. Can you give me an example of what you'd like to see?
<DarkDrgn2k[m]>
i have an IP address of a node that is running IPFS
<DarkDrgn2k[m]>
i woul dlike to use swarm connect to connect to it
nt80 has quit [Quit: Leaving]
<DarkDrgn2k[m]>
however i need it's ID to be able to do this
<DarkDrgn2k[m]>
(context: we have a script to peer with neigbouring IPFS server on CJDNS under conditions where bootstrap fails, however right now we have to expose the ID using HTTP ,i wanted to see if there was a way of doing this without requireing additional software on the remote node)
sammacbeth has quit [Ping timeout: 246 seconds]
sammacbeth has joined #ipfs
<aschmahmann[m]>
The peer ID is a hash of a public key and that public key is used to secure communications between nodes. Nonethless, there are some ways to do this (e.g. in go-ipfs 0.4.20 you can swarm connect to a random peer via dnsaddr see: https://blog.ipfs.io/83-go-ipfs-0-4-20/). However, unless the peer ID is exposed you're going to be headed for security issues since your peer might get impersonated.
<DarkDrgn2k[m]>
rigth now we have to curl a nodeinfo.json file
<DarkDrgn2k[m]>
that we would expect to be on the remote node
el_rubio has quit [Ping timeout: 250 seconds]
<DarkDrgn2k[m]>
so an IPFS server that is NOT one of or neds would never be connected to
<DarkDrgn2k[m]>
(thats how we currently expose the ID)
<DarkDrgn2k[m]>
so i guess the question is
<DarkDrgn2k[m]>
how do you get a public key from a node?
el_rubio has joined #ipfs
<DarkDrgn2k[m]>
`/dnsaddr/bootstrap.libp2p.io `
<DarkDrgn2k[m]>
so are you saying that i can do this instead of an ipv6 addy?
sammacbeth has quit [Ping timeout: 248 seconds]
merethan has joined #ipfs
<aschmahmann[m]>
unless I'm mistaken, because all you're doing is running curl against an IP address looking for an ID before you connect you are already ok with any peer showing up at the end of that connection (yours, a malcious party that exposes the same HTTP API as you, etc.). If so then the dnsaddr type approach might be accepatble for you.
<DarkDrgn2k[m]>
yes, i am alittle confused about the attack vector on here
sammacbeth has joined #ipfs
<DarkDrgn2k[m]>
what would some one gain trying ot impersonate a node?
<aschmahmann[m]>
there are a number of possible issues that may/may not be relevant for your case. For example, if I know that I can make sure you bootstrap from me then I can give you lots of nodes that are controlled by me. This then gives me the ability to make you think that you've made data available to the IPFS network at large, but in reality I've censored out anything I feel like.
<DarkDrgn2k[m]>
ahh i see
<aschmahmann[m]>
none of these issues are necessarily deal breakers, it just makes the topic grey enough that it's not as simple as "it's secure enough" or "NEVER USE" which is why the release notes say "Please do not rely on this except for testing or unless you know what you’re doing" 😅
<DarkDrgn2k[m]>
i saw that yet,
<DarkDrgn2k[m]>
yes
<DarkDrgn2k[m]>
i guess in our model the idea is that we are peerig with direct neigbours on a mesh, so as long as we can peer with as many nodes as we can for bootstrap it should mitigate that
<DarkDrgn2k[m]>
uneless its some sort of MITM attack
<aschmahmann[m]>
It's worth looking at what you're currently doing with your HTTP API exposure and seeing if this is actually any worse. It might indicate either revisiting what you're doing, or deciding that the dnsaddr approach is just as large an attack surface.
<DarkDrgn2k[m]>
we have "trusted" boot strap service but on an offline network these bootstraps are ussless
<DarkDrgn2k[m]>
I think being able to peer easier in a scenario like that would be better then peering less
<DarkDrgn2k[m]>
taking the mesh idea a bit further i noticed mention of "important" nodes
<DarkDrgn2k[m]>
do you know if tehre is any way to weigh nodes in a swarm? like define certain nodes to be more interesting then others?
merethan has quit [Remote host closed the connection]
<DarkDrgn2k[m]>
im looking at IPFS in a mesh topology where physical links are more important then traditiona intenrnet. Specifically if you DONT need to use the backhaul on a mesh network to request the file, dont.
<DarkDrgn2k[m]>
where DHT trees in ipfs dont use physical proximiy/links
<DarkDrgn2k[m]>
im wondering if we could some how push some of this knowledge the mesh network has about which nodes are easier to access ten others into the IPFS tree
<DarkDrgn2k[m]>
IE: 2 laptops doing the IPFS live streem on the same network, both running IPFS, it would make sence for the laptop next to you NOT to go to china becuase its the closts "DHT TREE" vector but instread grab it form the guy next to you
<DarkDrgn2k[m]>
sorry lots of wierd high leven questions here about stuff that been rattling around in my brain :P
sammacbeth has quit [Quit: Ping timeout (120 seconds)]
sammacbeth has joined #ipfs
Jesin has joined #ipfs
<deltab>
DokterBob: does the node still respond to requests? just doesn't log them?
<aschmahmann[m]>
No worries, if I recall you can define nodes as more "interesting"/important to the libp2p connection manager by "tagging" them. This helps libp2p figure out which connections to drop if you start to get overloaded with connections.
<aschmahmann[m]>
into account. I'm not sure how IPFS/bitswap decides which peers it wants to retrieve data from, but there are some totally reasonable expectations to have here.
<aschmahmann[m]>
Also, to clarify there are two major operations happening in an IPFS data retrieval. One is going to the DHT and retrieving a list of peers that have advertised having your content, and the second is going to those peers and downloading the content. The DHT request can be slow and can be very hard to design with proximity in mind. However, the data retrieval can be designed to take proximity, available bandwidth, etc.
<aschmahmann[m]>
one idea here is that it might be ok to sacrifice a bit on latency to do the DHT walk if in return you received numerous peers that could help you download a large object. In this case the latency to start receiving data would eventually be dwarfed by increased throughput once you start downloading.
<DarkDrgn2k[m]>
hmm well i can ttell you "ipfs pin" is SOOOO painfully slow..
<DarkDrgn2k[m]>
its 3-6 times faster to curl it against a gateway!
unashamed has joined #ipfs
el_rubio has quit [Ping timeout: 248 seconds]
<aschmahmann[m]>
DarkDrgn2k: For sure. Not saying that it works like that yet, but there are some good folks working on it so there's reason to be optimistic. If you look through some of the IPFS team's management/planning info (https://github.com/ipfs/team-mgmt and the ongoing PRs) you'll see that the painfully obvious issues are definitely being worked on.
<aschmahmann[m]>
Anyhow, got to head out now, but please PM me or reach out on github if you've got more questions (or want to be an improved IPNS over PubSub guinea pig)
<DarkDrgn2k[m]>
thanks so much :)
<DarkDrgn2k[m]>
appreciate the update
reit has quit [Ping timeout: 252 seconds]
Nd-969-M has joined #ipfs
unashamed has quit [Quit: Leaving]
Papa_Alpaka has joined #ipfs
sammacbeth has quit [Ping timeout: 252 seconds]
clemo has quit [Ping timeout: 252 seconds]
KempfCreative has quit [Ping timeout: 252 seconds]
sammacbeth has joined #ipfs
el_rubio has joined #ipfs
el_rubio has quit [Read error: Connection reset by peer]
Ai9zO5AP has quit [Quit: WeeChat 2.4]
cyfex has quit [Ping timeout: 264 seconds]
cyfex has joined #ipfs
alexgr has joined #ipfs
Adbray has quit [Disconnected by services]
Adbray_ has joined #ipfs
__jrjsmrtn__ has joined #ipfs
_jrjsmrtn has quit [Ping timeout: 244 seconds]
clemo has joined #ipfs
_nkls_ has quit [Remote host closed the connection]
themsay has joined #ipfs
plexigras has quit [Ping timeout: 248 seconds]
_nkls_ has joined #ipfs
_nkls_ has quit [Remote host closed the connection]
_nkls_ has joined #ipfs
warner has joined #ipfs
foxcpp has quit [Quit: Looks like my relay decided to commit suicide]
grorg[m] has left #ipfs ["User left"]
foxcpp has joined #ipfs
_nkls_ has quit [Remote host closed the connection]
themsay has quit [Quit: Leaving]
_nkls_ has joined #ipfs
_nkls_ has quit [Ping timeout: 250 seconds]
redfish has quit [Ping timeout: 258 seconds]
el_rubio has joined #ipfs
xcm has quit [Read error: Connection reset by peer]
xcm has joined #ipfs
mowcat has joined #ipfs
mowcat has quit [Max SendQ exceeded]
mowcat has joined #ipfs
yosafbridge has quit [Quit: Leaving]
redfish has joined #ipfs
_nkls_ has joined #ipfs
yosafbridge has joined #ipfs
_nkls_ has quit [Ping timeout: 246 seconds]
spinza has quit [Quit: Coyote finally caught up with me...]
m_tadeu has joined #ipfs
m_tadeu_ has quit [Ping timeout: 248 seconds]
alexgr has quit [Quit: Leaving]
_nkls_ has joined #ipfs
_nkls_ has quit [Remote host closed the connection]