lie to server about md5sum of a file

[babadoctor]

I want to upload very large files to a server, but according to some convention (i dont remember which one, it might be multipart data), it requires me to calculate an md5 hash of the file before uploading, and then the server checks to see if the md5 sum i provided is correct. if not the upload is aborted

 

is there any way I can "lie" to the server and tell it that it has the right hash?

 

or somehow calculate the hash of the file incrementally as it is created?

 

the file in question is created over a period of 12 hours, and data is added to it. is there any way to calculate the hash incrementally so the computer does not need  to do it all at once?

 

hashing a 50GB file is too much for the computer hosting it to do, so any support would be appreciated

[straight_stewie]

6 hours ago, babadoctor said:

is there any way I can "lie" to the server and tell it that it has the right hash?

I sure hope not. That'd completely undermine the point of hashes in the first place.

[Eigenvektor]

7 hours ago, babadoctor said:

or somehow calculate the hash of the file incrementally as it is created?

Depends on how you're creating the file. To calculate the hash, the file is read from beginning to end and fed through the hash algorithm. If you're creating the file programmatically you can absolutely generate the hash while you're writing it to a file, by feeding the bits through the hash digest as you're writing the bits to disk.

 

For example, the server can do this while it is receiving the file. As the file is written to disk, it can also send the stream through the hash algorithm at the same time. Unfortunately the sender can't do that because you need to present the MD5 to the server as a header before the file is sent, otherwise you could generate it as you're going over the file to send it.

[vorticalbox]

3 hours ago, Eigenvektor said:

Unfortunately the sender can't do that because you need to present the MD5 to the server as a header before the file is sent, otherwise you could generate it as you're going over the file to send it.

you could however create the hash as you create the file.

[Eigenvektor]

10 hours ago, vorticalbox said:

you could however create the hash as you create the file.

Yes, which mentioned in the first paragraph of my post