Cloudflare goes InterPlanetary - Cloudflare announces IPFS Gateway

[matrix07012]

Cloudflare goes InterPlanetary

Cloudflare announces IPFS Gateway

 

Quote

Today we’re excited to introduce Cloudflare’s IPFS Gateway, an easy way to access content from the the InterPlanetary File System (IPFS) that doesn’t require installing and running any special software on your computer. We hope that our gateway, hosted at cloudflare-ipfs.com, will serve as the platform for many new highly-reliable and security-enhanced web applications. The IPFS Gateway is the first product to be released as part of our Distributed Web Gateway project, which will eventually encompass all of our efforts to support new distributed web technologies.

 

Quote

Quick Primer on IPFS

 

Usually, when you access a website from your browser, your browser tracks down the origin server (or servers) that are the ultimate, centralized repository for the website’s content. It then sends a request from your computer to that origin server, wherever it is in the world, and that server sends the content back to your computer. This system has served the Internet well for decades, but there’s a pretty big downside: centralization makes it impossible to keep content online any longer than the origin servers that host it. If that origin server is hacked or taken out by a natural disaster, the content is unavailable. If the site owner decides to take it down, the content is gone. In short, mirroring is not a first-class concept in most platforms (Cloudflare’s Always Online is a notable exception).

The InterPlanetary File System aims to change that. IPFS is a peer-to-peer file system composed of thousands of computers around the world, each of which stores files on behalf of the network. These files can be anything: cat pictures, 3D models, or even entire websites. Over 5,000,000,000 files had been uploaded to IPFS already.

IPFS vs. Traditional Web

There are two key differences between IPFS and the web as we think of it today.

The first is that with IPFS anyone can cache and serve any content—for free. Right now, with the traditional web, most typically rely on big hosting providers in remote locations to store content and serve it to the rest of the web. If you want to set up a website, you have to pay one of these major services to do this for you. With IPFS, anyone can sign up their computer to be a node in the system and start serving data. It doesn’t matter if you’re working on a Raspberry Pi or running the world’s biggest server. You can still be a productive node in the system.

 

The second key difference is that data is content-addressed, rather than location-addressed. That’s a bit of a subtle difference, but the ramifications are substantial, so it’s worth breaking down.

Currently, when you open your browser and navigate to example.com, you’re telling the browser “fetch me the data stored at example.com’s IP address” (this happens to be 93.184.216.34). That IP address marks where the content you want is stored in the network. You then send a request to the server at that IP address for the “example.com” content and the server sends back the relevant information. So at the most basic level, you tell the network where to look and the network sends back what it found.

IPFS turns that on its head.

With IPFS, every single block of data stored in the system is addressed by a cryptographic hash of its contents, i.e., a long string of letters and numbers that is unique to that block. When you want a piece of data in IPFS, you request it by its hash. So rather than asking the network “get me the content stored at 93.184.216.34,” you ask “get me the content that has a hash value of QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy.” (QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy happens to be the hash of a .txt file containing the string “I’m trying out IPFS”).

How is this so different?

Remember that with IPFS, you tell the network what to look for, and the network figures out where to look.

 

Quote

How exactly do you access content on IPFS?

Now that we’ve walked through all the details of what IPFS is, you’re probably wondering how to use it. There are a number of ways to access content that’s been stored in the IPFS network, but we’re going to address two popular ones here. The first way is to download IPFS onto your computer. This turns your machine into a node of the IPFS network, and it’s the best way to interact with the network if you want to get down in the weeds. If you’re interested in playing around with IPFS, the Go implementation can be downloaded here.

But what if you want access to content that’s stored on IPFS without the hassle of operating a node locally on your machine? That’s where IPFS gateways come into play. IPFS gateways are third-party nodes that fetch content from the IPFS network and serve it to you over HTTPS. To use a gateway, you don’t need to download any software or type any code. You simply open up a browser and type in the gateway’s name and the hash of the content you’re looking for, and the gateway will serve the content in your browser.

Say you know you want to access the example.txt file from before, which has the hash QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy, and there’s a public gateway that is accessible at https://example-gateway.com

To access that content, all you need to do is open a browser and type

https://example-gateway.com/ipfs/QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy

and you’ll get back the data stored at that hash. The combination of the /ipfs/ prefix and the hash is referred to as the file path. You always need to provide a full file path to access content stored in IPFS.

 

What can you do with Cloudflare’s Gateway?

At the most basic level, you can access any of the billions of files stored on IPFS from your browser. But that’s not the only cool thing you can do. Using Cloudflare’s gateway, you can also build a website that’s hosted entirely on IPFS, but still available to your users at a custom domain name. Plus, we’ll issue any website connected to our gateway a free SSL certificate, ensuring that each website connected to Cloudflare's gateway is secure from snooping and manipulation. For more on that, check out the Distributed Web Gateway developer docs.

A fun example we’ve put together using the Kiwix archives of all the different StackExchange websites and build a distributed search engine on top of that using only IPFS. Check it out here.

 

My experience with IPFS so far hasn't been that great as it frequently suffers from slowdowns, so cache like Cloudflare's sounds great. 

 

Sauce: https://blog.cloudflare.com/distributed-web-gateway/ / https://archive.fo/Q1HDS

[Car712]

Dont laugh at me, but when I saw Interplanetary, I thought they were like running servers on satellites or something now LMAO

 

Then again, the only people that would benefit from that would be NASA or other countries space agencies

[Senzelian]

"A peer-to-peer hypermedia protocol to make the web faster, safer, and more open."

 

Safer and more open at the same time? Sure...  

[Ginger_]

This is weird... 

 

Still, it's gonna rely on people hosting so, Im not sure how well it'll take off

[Brooksie359]

1 hour ago, matrix07012 said:

Cloudflare goes InterPlanetary

Cloudflare announces IPFS Gateway

 

 

 

 

My experience with IPFS so far hasn't been that great as it frequently suffers from slowdowns, so cache like Cloudflare's sounds great. 

 

Sauce: https://blog.cloudflare.com/distributed-web-gateway/ / https://archive.fo/Q1HDS

How the hell am I supposed to memorize a randomized string of text that corresponds to the content I want to look at? I guess a new type of search engine is required?

[ZacoAttaco]

1 hour ago, matrix07012 said:

To access that content, all you need to do is open a browser and type

https://example-gateway.com/ipfs/QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy

It seems interesting, although it is not 'simple' to 'type' that address. I don't like the idea of the program turning your machine into another node, although it seems there is another version of the program to avoid this.

 

Who knows where this goes? Really interesting read though.

[jagdtigger]

2 minutes ago, ZacoAttaco said:

I don't like the idea of the program turning your machine into another node

In order to be truly decentralised it has to do that... But im afraid it wont be around for long, there are many ppl, comapny, agency, etc who do not want an internet that cannot be controlled.

[JoseGuya]

So... Like I2P?

[LAwLz]

11 hours ago, JoseGuya said:

So... Like I2P?

I haven't read the technical details yet but that's what it sounds like to me too.

And in that case I think we all know what IPFS is going to be used for... Child porn. Child porn everywhere...

 

Other than that drawback, it seems good. 

[mr moose]

12 hours ago, jagdtigger said:

In order to be truly decentralised it has to do that... But im afraid it wont be around for long, there are many ppl, comapny, agency, etc who do not want an internet that cannot be controlled.

I think you might be surprised at how long this, or ideas like it, will be around,  everything is heading down the distributed load/P2P path.  Blockchain is being implemented in more and more things, World leaders are looking at distributed networks to solve logistics and centralization issues.

 

I am more scared of my ability to get my head around the enormity of technology and understand when and how something is secure than I am about oppressive identities turning the world into some sort of soylent green/hunger games mashup.

[jagdtigger]

2 minutes ago, mr moose said:

I think you might be surprised at how long this, or ideas like it, will be around,  everything is heading down the distributed load/P2P path.  Blockchain is being implemented in more and more things, World leaders are looking at distributed networks to solve logistics and centralization issues.

Even if it takes off it wont be totally decentralized like how they want it. For one the copyright mafia wouldnt want a place where they cannot enforce their obsolete business model, and acronym agencies wouldnt like it either since they could not trace stuff that easily. Not to mention governments who like to censor stuff as they see fit.

[mr moose]

1 minute ago, jagdtigger said:

Even if it takes off it wont be totally decentralized like how they want it. For one the copyright mafia wouldnt want a place where they cannot enforce their obsolete business model, and acronym agencies wouldnt like it either since they could not trace stuff that easily. Not to mention governments who like to censor stuff as they see fit.

I don't share your outlook.  People have been afraid of those things for as long as we have been able to communicate.  There is nothing new under the sun and nothing seems to ever come of it, because when it does it doesn't seem to last.  Every empire falls, every dictator dies and the only thing that seems to be growing and expanding is true democratic freedom and standards of living.

[jagdtigger]

1 minute ago, mr moose said:

I don't share your outlook.

Because you look at it ideally and not realistically. There is too much ppl, corp, and government who do not want anything that cannot be controlled by them. Its just how the world is, true democracy will never come because there will be always ppl who will exploit the system.

[mr moose]

4 minutes ago, jagdtigger said:

Because you look at it ideally and not realistically. There is too much ppl, corp, and government who do not want anything that cannot be controlled by them. Its just how the world is, true democracy will never come because there will be always ppl who will exploit the system.

no, I look at it historically, and with 20+ years of international/social/political observation.

Not only is the entire population heading for better health, but we are all heading to a better standard of living and the number of democracies is increasing.  Even China is heading toward a democratic existence.  Look at how they were 40 years ago, then 20 years ago versus now.   Even NK is starting to talk,  Beirut looks nothing like it used, the IRA don;t bomb the shit out of everything every second day, Gay people can get married (there was once upon a time that that would cost you your job and even your life),  Russia technically  not a Communist country and things do seem to be improving.  Just because everything is not looking roses now doesn't mean it is getting worse, the fact is things are getting better now in most countries.  So before you point to atrocities in country X or shit things the the ABC agency in the US has done, I am not claiming everything is perfect, I am just saying that on the scale of historic evidence (even just in my time of 40 years) things around the world are improving and heading toward a free democratic state.

[MyName13]

2 hours ago, mr moose said:

everything is heading down the distributed load/P2P path

.  Blockchain is being implemented in more and more things,

World leaders are looking at distributed networks to solve logistics and centralization issues.

What everything?

Where is blockchain implemented?

Which world leaders are looking at what and what for?

[GOTSpectrum]

2 hours ago, jagdtigger said:

Because you look at it ideally and not realistically. There is too much ppl, corp, and government who do not want anything that cannot be controlled by them. Its just how the world is, true democracy will never come because there will be always ppl who will exploit the system.

You say that but TOR project is still running strong... 

[mr moose]

50 minutes ago, MyName13 said:

What everything?

the future everything

 

50 minutes ago, MyName13 said:

Where is blockchain implemented?

boeing use blockchain for there assembly line, medicalchain use blockchain to store health records, one belt one road is slated to use a block chain foundation for it's logistics (becasue it is decentralized and no one country or company can control it), A short google brings up these:

 

https://www.huffingtonpost.com/ameer-rosic-/5-blockchain-applications_b_13279010.html

https://hbr.org/2017/01/the-truth-about-blockchain

https://futurism.com/blockchain-future-pros-cons/

 

there is a lot going on and it is considered a very good technology for many applications. 

50 minutes ago, MyName13 said:

Which world leaders are looking at what and what for

Chinese government and everyone across the one road is looking at it seriously,  Australia is putting money into setting up blockchain standards so it can be used for government and public service.

 

https://www.intheblack.com/articles/2018/08/22/how-governments-using-blockchain-technology

 

plus

https://hackernoon.com/blockchain-for-government-41e3b097356d

https://www.computerworlduk.com/galleries/applications/how-governments-are-using-blockchain-3680393/

 

 

I know people like to play the what what game and ask for citations, but this really is a big thing and is happening all around the world.  Interest in secure decentralized ledgers is high and blockchain is just that.

[toor]

 

Kinda surprised there isn't more news about this, it's a pretty big deal.  IPFS is not the same as i2p though.  i2p was designed from the ground up to be anonymous and private.  IPFS is more like a torrent but with more options (stream multiplexing for example)..   

 

Both use hashes to find data.  Unlike i2p, ipfs doesn't at least atm have a notion of SSK, HSK, KSK, etc nor is the data at rest  - and this is the important part - encrypted  by default.  There's nothing to stop you from doing so (connections of course, are).  You also don't have to be part of any particular network, it's a bit more like LAFS.   

 

As for performance,   I have no problem hitting 100+M/s on cache hot data.   I'm sure Cloudflare is still tweaking things though.   I'd imagine they are ingesting a whole lot of data given some of the scientific dumps are TBs in size.  What they retain is another matter.

 

 

tl;dr - I'd love if LMG or Floatplane tested with even a small dataset.  Could be media or some encrypted archives.   Doubt they'd have a hard time finding peers willing to help.

[Sniperfox47]

On 9/17/2018 at 4:32 PM, JoseGuya said:

So... Like I2P?

They're kind of different layers. You can run IPFS over I2P just like you can run it over HTTPS, but it's more of a distributed network filesystem (hence the name) than it is a traditional web network.

On 9/18/2018 at 6:41 AM, MyName13 said:

What everything?

Where is blockchain implemented?

Which world leaders are looking at what and what for?

I mean Canada is looking to adopt it pretty heavily for education programs, government documentation and transparency, government accountability (since it's so hard to go back and edit something that's already been submitted), and bussiness programs. The NRC has run a number of different trials that have been getting larger and larger in scope and a couple of those trials have spawned into full-blown programs already and a few others are being considered for a full implimentation.