Read YAML file using Golang and store data in boltdb

In this tutorial, we'll use GO to read a YAML file and save its contents in boltdb.
The YML file is commonly used to hold configuration information for any server or process. For example, in Ansible, we save deployment information in the YML file, which is extremely easy to read and comprehend.

 

Boltdb is a Golang library that is used to store key-value based data. It is highly fast and supports transactions, so if any input or update is erroneous, we can undo the change.

"github.com/boltdb/bolt"

I'm going to utilise a third-party library to read the yml file in this application, which is:

"gopkg.in/yaml.v2"

The documentation for this library is excellent and extremely simple to use; I strongly advise you to use it in your project. A higher version is available on github, but I have not used it yet; if I do, I will let you know.

For this article, I used the internet to get a yaml file with cities and a lengthy store.

Consider the following scenario:

 

A1: { name: 'Anonymous Proxy', latitude: 0, longitude: 0 }
A2: { name: 'Satellite Provider', latitude: 0, longitude: 0 }
AD: { name: Andorra, latitude: '42.546245', longitude: '1.601554' }

Our goal is to save these records in the bolt database depending on their key.

I used the map function and created a struct based on the object in the example. I developed the sturct with the aid of the yaml library so that while unmarshaling the data, it stores the data at its exact position.

type ymlDetail struct {
    Name       string `yaml:"name"`
    Latitude   string `yaml:"latitude"`
    Longtitude string `yaml:"longitude"`
}

In you case just replace your key in yaml section so while unmarshalling its store the data based the member variable column name.

Below is the full working code, you can check and if required let me know in comment if you have doubts.

 

package main

import (
	"encoding/json"
	"fmt"
	"io/ioutil"
	"log"

	"github.com/boltdb/bolt"
	"gopkg.in/yaml.v2"
)

var (
	db *bolt.DB
)

func init() {
	var err error
	db, err = bolt.Open("my.db", 0600, nil)
	if err != nil {
		log.Fatal(err)
	}
	//required to create similar to table where all our data stored
	createBucket()
}

type ymlDetail struct {
	Name       string `yaml:"name"`
	Latitude   string `yaml:"latitude"`
	Longtitude string `yaml:"longitude"`
}

//CreateBuck similar to table where we store all our table
func createBucket() {
	/* 	tx, err := db.Begin(true)
	   	if nil != err {
	   		log.Fatalf("Error while start transaction: %s", err.Error())
	   	} */
	db.Update(func(tx *bolt.Tx) error {
		var err error
		_, err = tx.CreateBucketIfNotExists([]byte("Country"))
		if nil != err {
			log.Printf("Create bucket: %s", err.Error())
		}
		return err
	})
}

//for safely store key value inside the bolt db
func storeDataInsideBoltDB(key string, values ymlDetail) {
	byts, _ := json.Marshal(values)
	db.Update(func(tx *bolt.Tx) error {
		bucket := tx.Bucket([]byte("Country"))
		err := bucket.Put([]byte(key), []byte(byts))
		if nil != err {
			log.Fatalf("Error while storing data: %s", err.Error())
		}
		return err
	})
}

func validateKeyStoreInsideTheDB(key string) {
	db.View(func(tx *bolt.Tx) error {
		bucket := tx.Bucket([]byte("Country"))
		v := bucket.Get([]byte(key))
		fmt.Printf("The answer is: %s\n", string(v))
		return nil
	})
}

func readYMLfile(fileName string) {
	yamlData, err := ioutil.ReadFile(fileName)
	if nil != err {
		log.Fatalf("Error while Reading file:%s", err.Error())
	}
	countryDetails := make(map[string]ymlDetail)
	err = yaml.Unmarshal(yamlData, &countryDetails)
	if nil != err {
		log.Fatalf("Error while Unmarshal file:%s", err.Error())
	}
	for key, country := range countryDetails {
		storeDataInsideBoltDB(key, country)
	}
}

func main() {
	defer db.Close()
	readYMLfile("countries.yml")
	validateKeyStoreInsideTheDB("ML")
}


 

 

Copy the content and also store the given input data in side the country.yml file and run the program, you will see the result based on the given input.

There is  a my,db file will create in your local directory.