The Kit project in Go is the common project containing all the standard libraries or packages used across all the Go projects in the organization.

Motivation

I attended GopherCon India 2017, there was a talk on “Package Oriented Design In Go” by William Kennedy. In that talk, William explained some really important and thoughtful design principles which We can apply in our day to day life, while writing Go. Hence, I wanted to apply these design philosophies to the Go projects in which I have been working on as a part of OpenEBS project. I learnt a good practice of having a Go Kit project at the organization level from William’s talk.

What is the Kit Project?

The Kit project in Go is the common project containing all the standard libraries or packages used across all the Go projects in the organization. Packages in the Kit project should follow design philosophies.

Need for a Kit project

Sometimes, We write same Go packages again and again to do the same task at different levels in the different Go projects under the same organization. For example, we write custom logger package in the different Go projects. If the custom logger package is same across the organization and can be reused by simply importing it, then this custom logger package is the perfect fit for Kit project. You can sense how much time and cost it will save for us when we have a Kit project.

How to convert existing projects to have “kit”

Maya is the kit project in the progress. I will walk through our journey of creating a Kit project called maya for OpenEBS organization from existing Go projects. At OpenEBS, as a open source and growing Go project, We value Go principles and We try hard to leverage Go’s offerings. Maya is the Kit project for the Application projects like maya-apiserver, maya-storage-bot etc. Maya contains all the kubernetes & nomad API’s, common utilities etc. needed for development of maya-apiserver and maya-storage-bot. In the near future, we are trying to push our custom libraries to maya. So that, it will become a promising Go kit project for OpenEBS community.

We have specifically followed the package oriented design principles in Go to create maya as a kit project.

• Usability

  We moved common packages such as orchprovider, types, pkg to maya from maya-apiserver. These packages are very generic and can be used in most of the Go projects in OpenEBS organization. Brief details about new packages in Maya.

  • Orchprovider : orchprovider contains packages of different orchestrators such as kubernetes and nomad.

  • types: types provides all the generic types related to orchestrator.

  • pkg: pkg contains packages like nethelper, util etc.

  • volumes: volumes contain packages related to volume provisioner and profiles.

• Purpose

  Packages in the Kit project should be purposeful. These packages must provide, not contain. In maya, we have packages like types, orchprovider, volumes etc. name of these packages suggests the functionality provided by them.

• Portability

  Portability is important factor for packages in kit project. Hence, we are making maya in such a way that it will be easy to import and use in any Go project. Packages in the Maya are not single point of dependency and all the packages are independent of each other.

Project structure for the Maya as Kit project

• Without having Maya as a kit project:

  • Project structure for the Maya

  
    ├── buildscripts
    │   └── docker
    ├── command
    ├── docs
    ├── example
    ├── scripts
    │   └── config
    └── templates
  

  • Project structure for the Maya-apiserver

    ├── buildscripts
    │   └── docker
    ├── cmd
    ├── docs
    ├── lib
    │   ├── api
    │   │   └── v1
    │   ├── artifacts
    │   │   ├── docker
    │   │   ├── docker-compose
    │   │   └── k8s
    │   ├── config
    │   ├── flaghelper
    │   ├── loghelper
    │   ├── mockit
    │   │   └── etcmayaserver
    │   ├── nethelper
    │   ├── orchprovider
    │   │   ├── k8s
    │   │   └── nomad
    │   ├── profile
    │   │   ├── orchprovider
    │   │   └── volumeprovisioner
    │   ├── server
    │   ├── util
    │   └── volumeprovisioner
    │       └── jiva
    └── proposals
  

• Maya as a kit project

  • Project structure for the Maya (Kit Project)

  
    ├── buildscripts
    │   └── docker
    ├── command
    ├── docs
    ├── example
    ├── orchprovider
    │   ├── k8s
    │   │   └── v1
    │   └── nomad
    │       └── v1
    ├── pkg
    │   ├── nethelper
    │   └── util
    ├── scripts
    │   └── config
    ├── templates
    ├── types
    │   └── v1
    │       └── profile
    │           └── orchestrator
    └── volumes
        ├── profile
        │   └── volumeprovisioner
        └── provisioner
            └── jiva
  

  • Project structure for the Maya-apiserver (Application Project)

    ├── buildscripts
    │   └── docker
    ├── cmd
    ├── docs
    ├── lib
    │   ├── artifacts
    │   │   ├── docker
    │   │   ├── docker-compose
    │   │   └── k8s
    │   ├── config
    │   ├── flaghelper
    │   ├── loghelper
    │   └── server
    └── proposals
  

Example usage of maya kit project in maya-apiserver

Maya-apiserver uses maya as a Kit project. Maya-apiserver exposes OpenEBS operations in form of REST APIs. This allows multiple clients e.g. volume related plugins to consume OpenEBS storage operations exposed by Maya API server.

• Need for orchestration providers in maya-apiserver

  Maya-apiserver has been designed keeping storage inside containers in mind. In other words, containerized storage. This leads to maya-apiserver using best of breed container orchestrators. Hence maya-apiserver abstracts the container related orchestration needs in form of orchprovider namespace. Therefore,to achieve above need maya-api server uses orchprovider package provided by Maya.

• Need for volumes in the maya-apiserver

  There can be multiple persistent volume provisioners e.g. jiva & cstor which has the details of workings of its volume. maya-apiserver tries to abstract these into volumes operations whose specifics will reside in individual namespaces i.e. jiva, cstor, etc. Hence, maya-apiserver will make use of volumes package from Maya to satisfy above requirements.

Conclusion

Go Kit project should contain packages which are usable, purposeful and portable. Go Kit projects will improve the efficiency of the organization at both human and code level.