ChanServ changed the topic of #crystal-lang to: The Crystal programming language | http://crystal-lang.org | Crystal 0.32.1 | Fund Crystal's development: http://is.gd/X7PRtI | GH: https://github.com/crystal-lang/crystal | Docs: http://crystal-lang.org/docs/ | API: http://crystal-lang.org/api/ | Gitter: https://gitter.im/crystal-lang/crystal
<FromGitter> <randiaz95> is it wierd that https://crystal-lang.org/api/master/Zip/CompressionMethod.html the Zip compression method is not really a compression method but an enum?
<FromGitter> <randiaz95> Essentially I am trying to get a string and write it into a zipped file.
<FromGitter> <Blacksmoke16> Method in this case means like the "algorithm" to use
gangstacat has quit [Quit: Ĝis!]
<FromGitter> <randiaz95> Yep, Figured it out, what stumped me on this is that i need to put a filename while adding, because I already gave the Writer a filename through the file object with extension zip.
<FromGitter> <randiaz95> essentially you need a filename with `.zip` and then a `.txt` whilst adding
<FromGitter> <randiaz95> Theres no way to just store string alone into a zip file.
<FromGitter> <tenebrousedge> the Zip file format only stores files, not strings, so you need to at least give it a name inside the zip file
<FromGitter> <randiaz95> nah, the add method inside of the ZipWriter object needs filename and string content
<FromGitter> <tenebrousedge> that doesn't contradict what I said
<FromGitter> <randiaz95> Sorry I know saying no is a painful word, specifically over the internet where I can't say it in a nice way lol..
gangstacat has joined #crystal-lang
<FromGitter> <randiaz95> I meant that the only way to create a new zip with the current API is with both a filename for the inner file and it's content. ( I am sure we can abstract the inner files name to some content, however it may cause problems for some people's applications , thus that is why one cannot do it without the inner filename... )
<FromGitter> <randiaz95> I think there should be a default value for that argument though and one can then choose to omit the inner filename if they want
<FromGitter> <tenebrousedge> I think you're probably alone in that opinion
HumanG33k has quit [Read error: Connection reset by peer]
<FromGitter> <randiaz95> right, my application doesn't matter what the inner filename is; I am sure most other applications do care about the inner filename.
<FromGitter> <tenebrousedge> you could probably write a compressed string to a file some other way, if the goal isn't necessarily to create a compressed filesystem
<FromGitter> <randiaz95> lol great deduction, I am trying to compress my application's file system so that I can spend less money on server hard disk.
<FromGitter> <randiaz95> Side hustles suck and I am not good at dev ops when I am at my day job.
<FromGitter> <tenebrousedge> hmmmm. what exactly are you compressing?
<FromGitter> <randiaz95> ok, I create dashboards through the website I made https://wombol.com
<FromGitter> <randiaz95> to checkout a demo use: demo as account and `Welcome.1` as the password
<FromGitter> <randiaz95> Essentially I have tons of social media data, amazon marketplace data, and website data feeding custom dashboards
<FromGitter> <j8r> if server hard disk is the topic, why don't directly compress it?
<FromGitter> <j8r> the whole filesystem?
<FromGitter> <randiaz95> So on every request I have to compress the entire filesystem?
<FromGitter> <Blacksmoke16> im assuming its data that is used, not just like archived stuff?
<FromGitter> <j8r> No, it automatically done @randiaz95
<FromGitter> <randiaz95> Thing is, the easiest way to do that is to use a RDBMS to autocompress data but RDBMS is too much time with manually dropped csvs and excels
<FromGitter> <randiaz95> no data that is used to feed the dashboards.
<FromGitter> <j8r> In any case there will have a little CPU penalty, of course
<FromGitter> <randiaz95> I miswrote Julien..
<FromGitter> <randiaz95> do you mean unzip the entire filesystem on request?
<FromGitter> <j8r> The whole `/` is compressed.
<FromGitter> <randiaz95> 1) o
<FromGitter> <randiaz95> Wow LMAO
<FromGitter> <randiaz95> God bless open source
<FromGitter> <j8r> This means, everything you put on it is automatically compressed
<FromGitter> <tenebrousedge> amen πŸ‘Ό
<FromGitter> <randiaz95> ROFL
<FromGitter> <randiaz95> So i would interact with the filesystem normally?
<FromGitter> <j8r> yes, transparent
<FromGitter> <randiaz95> File.read in crystal and with open("", 'r') as file in python?
<FromGitter> <randiaz95> wow
<FromGitter> <randiaz95> Amazing....
<FromGitter> <j8r> you can also compress RAM memory for saving
<FromGitter> <randiaz95> what?
<FromGitter> <j8r> yep, ZSWAP on Linux
<FromGitter> <j8r> that's somethings very common on macOS and Windows
<FromGitter> <randiaz95> for free?
<FromGitter> <randiaz95> like what are the drawbacks of compressing your ram? I know if you compress your file system it will take longer to load because of unzipping
<FromGitter> <randiaz95> I would have to check if these services in arch work in ubuntu
<FromGitter> <j8r> I think it would be comparable, or faster, than doing it yourself on your app
<FromGitter> <j8r> It's not Arch specific
<FromGitter> <j8r> that's very Linux specific
<FromGitter> <randiaz95> brilliant...
<FromGitter> <randiaz95> ok I will make my api File.read instead of using the zip library in crystal, thanks bro!
<FromGitter> <j8r> yep, furthermore zip is not always the best choice
<FromGitter> <j8r> @randiaz95 np :)
<FromGitter> <j8r> and for perf, usually you will want to use `File.open {}`, to avoid intermediary strings
HumanG33k has joined #crystal-lang
<FromGitter> <randiaz95> hmm, so File.open { |payload| puts payload} is less ram than File.read?
<FromGitter> <randiaz95> intermediary strings meaning it will copy the payload?
<FromGitter> <randiaz95> I was just moving the api from file.open to file.read because it was easier to read in one line lol.
<FromGitter> <randiaz95> File.read File.write
<FromGitter> <Blacksmoke16> it means it'll create a string just to puts it to STDOUT
<FromGitter> <Blacksmoke16> vs putsing the IO directly, saving the need to create that string
<FromGitter> <randiaz95> Perfect, I understand now. Sorry I don't like to not understand.
<FromGitter> <randiaz95> You guys are very helpful
<FromGitter> <Blacksmoke16> is best to check the method's you're using, if it accepts IO | String, is usually better to give it an IO if possible
<FromGitter> <randiaz95> Going straight to the bare metal is more intelligent
early has quit [Quit: Leaving]
duane has quit [Ping timeout: 260 seconds]
duane has joined #crystal-lang
early has joined #crystal-lang
f1refly has quit [Ping timeout: 252 seconds]
f1reflyylmao has joined #crystal-lang
lvmbdv has joined #crystal-lang
_whitelogger has joined #crystal-lang
_whitelogger has joined #crystal-lang
_whitelogger has joined #crystal-lang
_whitelogger has joined #crystal-lang
lvmbdv has quit [Ping timeout: 260 seconds]
_whitelogger has joined #crystal-lang
lvmbdv has joined #crystal-lang
lvmbdv has quit [Quit: Leaving]
snapcase has quit [Quit: leaving]
ht_ has joined #crystal-lang
livcd has joined #crystal-lang
snapcase has joined #crystal-lang
rohitpaulk has joined #crystal-lang
rohitpaulk has quit [Remote host closed the connection]
sorcus has joined #crystal-lang
alexherbo2 has joined #crystal-lang
alexherbo2 has quit [Quit: The Lounge - https://thelounge.chat]
alexherbo2 has joined #crystal-lang
alexherbo28 has joined #crystal-lang
alexherbo2 has quit [Ping timeout: 268 seconds]
<FromGitter> <j8r> ho, also it seems for some workload having a compressed filesystem can make your system quicker
<FromGitter> <j8r> because less disk IO due to smaller files
<FromGitter> <j8r> a bit related, I created a library to interact with LXD https://github.com/j8r/crystalxd ⏎ Still WIP
hightower3 has joined #crystal-lang
_whitelogger has joined #crystal-lang
_whitelogger has joined #crystal-lang
lvmbdv has joined #crystal-lang
fireglow has quit [Remote host closed the connection]
Human_G33k has joined #crystal-lang
HumanG33k has quit [Ping timeout: 260 seconds]
DTZUZO_ has quit [Read error: Connection reset by peer]
DTZUZO_ has joined #crystal-lang
fireglow has joined #crystal-lang
alexherbo283 has joined #crystal-lang
alexherbo28 has quit [Ping timeout: 260 seconds]
<FromGitter> <j8r> I prefer LXC over classic VMs if I need a Linux sandbox :)
<FromGitter> <tenebrousedge> docker <3
<FromGitter> <j8r> docker is only designed to run a single service
<FromGitter> <j8r> not a whole OS with the init system
<FromGitter> <tenebrousedge> uhhhh
<FromGitter> <tenebrousedge> docker uses LXC
<FromGitter> <tenebrousedge> and is almost exclusively used to run whole OSes
<FromGitter> <j8r> try to run systemd or OpenRC inside docker, you will see
<FromGitter> <j8r> you have to do hacks like this: https://hub.docker.com/r/jrei/systemd-debian
<FromGitter> <j8r> don't do this in a production environment πŸ’€
<FromGitter> <tenebrousedge> hacks like using an image designed for that purpose?
<FromGitter> <j8r> no, you have to create mounts with the underlying system, and run as privileged therefore
<FromGitter> <j8r> so the container isn't really isolated anymore
<FromGitter> <j8r> Docker is designed to be stateless (we have to create mounts to keep data), LXD/LXC is both
<FromGitter> <tenebrousedge> Docker is designed to be stateless for very good reasons
<FromGitter> <j8r> maybe, but that's not always adapted to all use cases
<FromGitter> <tenebrousedge> nor is any other technology
<FromGitter> <j8r> that's why LXC/LXD is good for a VM-like environment :)
<FromGitter> <j8r> and Docker for application development
<FromGitter> <tenebrousedge> why bother running a Linux VM on a Linux host?
<FromGitter> <j8r> to not pollute it.
<FromGitter> <tenebrousedge> with?
<FromGitter> <j8r> with databases, applications, etc. And it is to test other distributions
<FromGitter> <j8r> I want to test something that needs root - I don't want to mess up my system, sure
<FromGitter> <tenebrousedge> what would you test that needs root?
<FromGitter> <j8r> Ansible for instance
<FromGitter> <j8r> or database installation
<FromGitter> <tenebrousedge> what risk is there in installing either of those?
<FromGitter> <j8r> because if it doesn't work, my system can be impacted
<FromGitter> <j8r> Ansible modify system used, ntp, hosts, init services
<FromGitter> <tenebrousedge> and in your testing, did you find that purging the package reverted all those changes?
<FromGitter> <j8r> it wont, it's not only about packages
<FromGitter> <j8r> there are always configurations files that will stay
<FromGitter> <tenebrousedge> they shouldn't if the package is purged
<FromGitter> <j8r> Ansible is about installation, not uninstallation
<FromGitter> <j8r> It ensures everything is setup
<FromGitter> <j8r> sure you can uninstall with it
<FromGitter> <j8r> but if tasks modify files, you will have to do it manually to revert
<FromGitter> <j8r> that's always good to have a safe, reproducible, controlled sandbox
<FromGitter> <j8r> it avoids to have "it works on my PC", "but not on yours" - lol
<FromGitter> <tenebrousedge> that's sort of the point of ansible though, to ensure that the system reaches a known state
<FromGitter> <j8r> indeed, and we don't want to have this state locally
<FromGitter> <j8r> that's why having a VM s close as the production is good
alexherbo283 has quit [Quit: The Lounge - https://thelounge.chat]
<FromGitter> <tenebrousedge> okay. I'm probably not going to be in a hurry to run multiple services on the same machine though
<FromGitter> <j8r> for instance, databases doesn't work very well in Docker, even less on K8S
<FromGitter> <j8r> That's why we stick with the classic way to do for this
<FromGitter> <tenebrousedge> you're probably alone in thinking databases don't work well in Docker/K8s
<FromGitter> <j8r> no, I don't think so
<FromGitter> <j8r> virtualization layers add IO latency
<FromGitter> <j8r> and that's fairly important for databases
<FromGitter> <j8r> furthermore, a database is stateful
<FromGitter> <tenebrousedge> so you have a volume
<FromGitter> <j8r> mounted locally? This break the whole point of K8S
<FromGitter> <j8r> and you can no longer destroy the node like this
<FromGitter> <j8r> or perhaps using CEPH, but it adds latency, too
<FromGitter> <j8r> you are right, you can, but IMO it's not as good as for stateless applications
<FromGitter> <Daniel-Worrall> Docker is great for stateful applications. You just need to extrapolate the state out using mounting in various ways. You can have multiple database processes accessing the same database information, but you should still introduce sharding for very large databases
<FromGitter> <Daniel-Worrall> Mounts don't have to be local
<FromGitter> <j8r> but then, what's the point of K8S?
<FromGitter> <j8r> a dedicated hosts for the databases will perfectly do the job
<FromGitter> <Daniel-Worrall> What do you mean? k8s uses mounts
<FromGitter> <j8r> and will have optimum performace
<FromGitter> <j8r> For me local mounts are not an option
<FromGitter> <j8r> and remote mounts add latency.
<FromGitter> <Daniel-Worrall> For production, sure. Local mounting is used mainly for development
<FromGitter> <Daniel-Worrall> remote mounts are usually found in the same datacenter, they're only remote in the context of the host
<FromGitter> <j8r> Some databases can already use sharding/replication natively (ElasticSearch, PostgreSQL, Redis) ⏎ No need to have a redundant FS like Ceph then
<FromGitter> <j8r> I just find the advantages less interesting for this kind of apps
<FromGitter> <j8r> What we have do with prometheus is running is on a dedicated node, alone, with a local mounted volume
<FromGitter> <j8r> Because else, there is performance impact
<FromGitter> <tenebrousedge> you've benchmarked the alternatives?
<FromGitter> <j8r> On Amazon, the best is to use the local EBS storage
<FromGitter> <j8r> anyway, it not critical data, that's fine
<FromGitter> <j8r> other topic, Alpine 3.11 :)
<FromGitter> <j8r> there is Crystal 0.31.1 on it
<FromGitter> <j8r> Need to do a PR for 0.32.1
<FromGitter> <j8r> I really want to solve https://github.com/crystal-lang/crystal/issues/8080, but I struggle :(
ht_ has quit [Remote host closed the connection]
Human_G33k has quit [Remote host closed the connection]
ht_ has joined #crystal-lang
ua has quit [Ping timeout: 265 seconds]
ua has joined #crystal-lang
duane has quit [Read error: Connection reset by peer]
duane has joined #crystal-lang
f1reflyylmao has quit [Quit: bye fags]
f1refly has joined #crystal-lang
<FromGitter> <nsuchy> Does Crystal have something like this yet? https://devblogs.microsoft.com/typescript/announcing-typescript-3-7/#optional-chaining
<FromGitter> <Blacksmoke16> `.try?`
<FromGitter> <tenebrousedge> ^
<FromGitter> <Blacksmoke16> https://crystal-lang.org/api/master/Object.html#try(&)-instance-method
<FromGitter> <nsuchy> so that's the equivalent?
<FromGitter> <tenebrousedge> yes
<FromGitter> <randiaz95> Lol, its so funny seeing MSFT using `ish`
<FromGitter> <randiaz95> my thought is Microsoft thought highly of themselves and always used intense language
<FromGitter> <randiaz95> I personally stick to begin rescue end
<FromGitter> <tenebrousedge> `try` doesn't swallow exceptions, just `nil`
<FromGitter> <randiaz95> interesting...
<FromGitter> <tenebrousedge> it's most useful for dealing with nilable types
<FromGitter> <tenebrousedge> within the `try` block, the value is guaranteed to be non-`nil`
<FromGitter> <randiaz95> nilable type is something like Int8|nil ?
<FromGitter> <tenebrousedge> or `arr.find { ... }.try do |value|`
<FromGitter> <randiaz95> interesting, so find returns nil if it doesn't find anything
<FromGitter> <tenebrousedge> yes, and also whatever method you call in the block doesn't yell at you because it's not defined on `Nil`
<FromGitter> <tenebrousedge> the alternative is `if(a = arr.find { ... ])`
<FromGitter> <tenebrousedge> you have to do an assignment and a test for falsity
<FromGitter> <j8r> I usually prefer doing `if value = arr.find { ... }.try` compared to `arr.find { ... }.try do |value|`
<FromGitter> <j8r> I find `try` is useful on single line statements
DTZUZO_ has quit [Read error: Connection reset by peer]
DTZUZO_ has joined #crystal-lang
ht_ has quit [Remote host closed the connection]
DTZUZO_ is now known as DTZUZO
sorcus has quit [Ping timeout: 260 seconds]
sorcus has joined #crystal-lang
DTZUZO has quit [Ping timeout: 240 seconds]
DTZUZO has joined #crystal-lang