Edit buffersize in golang

[manicottimuffin]

Hey everyone,

 

I am copying a file using Go and must make sure that the buffer size of my last block matches the source file exactly, not extending the new file any longer. The files must be identical. I have a working program, I only need help with changing the buffer size for the last block.

 

"One common mistake made by students is that they forget to consider the last block situation. Because a file can be of any size, the last block MIGHT NOT be filled to the very end. As a result, you need to write ONLY the number of bytes that are available in the last block."

 

I have the code working, but I can't figure out how to change the buffer size.

 

 

package main

import (
	"fmt"
	"io"
	"log"
	"os"
	"path/filepath"
)

//BSIZE is the size of the buffer
var BSIZE int64 = 4096

var err error = nil

func main() {

	if len(os.Args) != 3 {
		fmt.Printf("usage: %s source destination \n", filepath.Base(os.Args[0]))
		os.Exit(3)
	}

	//open the source file
	source, err := os.Open(os.Args[1])
	if err != nil {
		fmt.Println("Error opening file...")
		log.Fatal(err)
	}

	//check to make sure the destination file does not already exist
	_, err = os.Stat(os.Args[2])
	if err == nil {
		fmt.Println("Destination file already exists.")
		log.Fatal(err)
	}

	//create destination file
	destination, err := os.Create(os.Args[2])
	if err != nil {
		log.Fatal(err)
	}

	//actual copying of the file
	buf := make([]byte, BSIZE)
	for {
		n, err := source.Read(buf)
		if err != nil && err != io.EOF {
			log.Fatal(err)
		}
		if n == 0 {
			break
		}

		if _, err := destination.Write(buf[:n]); err != nil {
			log.Fatal(err)
		}
	}

	//close the files
	defer source.Close()
	defer destination.Close()

}

 

[mariushm]

A simple solution would be to initialize a variable with the file size of the file

Then, for as long as this variable is bigger than the buffer size or you did not reach the end of file, read a full buffer size worth of data and substract the amount from the variable.

When the value is below 4K but higher than 0 or you didn't reach the end of file, recreate the buffer variable with the remaining size or use a second buffer variable defined with the maximum size left to download.

 

Another option would be to use the slice() to create a second variable that holds only the amount of bytes you want.

something like

if n<BSIZE && n > 0 {

 max_size = n-1

 buf := buf.slice(0:max_size)

}

now your buf will be exact that many bytes .. but be careful if there's still more data to read, as you'll have to recreate the buffer to your BSIZE or continue reading n bytes or less.

[DevBlox]

You're not going to change the buffer size afterwards. You can slice it and get the length of the slice. Like so

len(buf[:n])

 The way you are doing it now is correct and good enough IMO, you don't need to do much more, well, depends on what you're trying to do.

 

I'd also encourage using 

io.Copy(dst, src)

whenever possible. It can do the entire "actual copying the file" in one line. Again, might not be what you need or want, just saying just in case you don't know this function.

 

EDIT: Probably should mention that when you slice the buffer you get a subset of that same buffer, laid out on the same memory. Basically, use properties of slices.