HashiCorp Vagrant started as a small project to make interacting with virtual machines for local development easy. A growing community of developers, operators, designers, and products rely on Vagrant. In pursuing the goal to continue improving Vagrant, we discovered a path that would allow Vagrant to be ported to Go and still support its Ruby-based features. In this post, we’ll discuss how migration to a Go code base will not only support developer environments of the past decade but also new development workflows, environments, and ecosystems.
The changes we are working on now are designed to allow you to run Vagrant with a client-server architecture, secure it better on different operating systems, and manage its configuration globally. They’ll also offer a more seamless plugin development experience by improving the portability of plugins and reducing the complexity of package management.
Over the next year, Vagrant 2.3 and 2.4 will not break compatibility promises of Vagrantfiles or plugin interfaces. Vagrant 3.0 will introduce new methods for configuration but retain tooling for continued compatibility of Vagrantfiles. This includes detection for Ruby Vagrantfiles and installation of compatibility helpers to minimize interruptions to user workflow. While we don't yet have specific timeframes, we will announce our plans for these new versions of Vagrant as they solidify over the next few months.
You can look forward to many new features, capabilities, and improvements made possible by these updates to Vagrant’s architecture:
The new architecture will allow you to run Vagrant on a remote host and secure actions on the machine. You will be able to install Vagrant on a resource-intensive machine and interact with it on a thin client, which will allow sharing of one Vagrant environment among multiple team members.
A new client-server model will facilitate easy sharing of configuration and resources while also giving administrators power over Vagrant’s privileged actions. This streamlines the collaboration experience offered by the
existing Vagrant Share plugin
. You will still be able to run Vagrant locally on a desktop machine and use the new security capabilities.
Vagrant runs as a user process that requires acceptance from a user to run privileged actions. You can work around request and acceptance on POSIX platforms using a
sudoers
file to define each Vagrant command with escalated privileges. Unfortunately, you need to update this file any time you change the privileged Vagrant commands. On Windows systems, you cannot use a
sudoers
file. Instead, you must confirm a User Account Control (UAC) prompt, which makes it difficult to run Vagrant in a headless context. Vagrant 3.0 will enable a privileged service to execute known and trusted commands from Vagrant and its plugins without the need for direct user interaction.
In addition to finer control of trusted commands, you will be able to use controls to prevent automatic evaluation of Ruby-based Vagrantfiles. You will also be able to write configuration in HashiCorp Configuration Language (HCL) to limit the potential for mistakes or risk arising from dynamic configuration. This prevents arbitrary code execution inline in Vagrantfiles.
Currently, you can reference only your own local configuration of Vagrant. Different users on the same system do not have access to the same configuration. A new, server-based approach will allow for management of a global Vagrant configuration, which you can also store in a database. This will fix
long-running issues
where provider-specific information results in lost configuration settings. Lost settings can result in unexpected and unintended system modifications.
Global configuration management helps organize Vagrant’s state and lays the foundation for adding a web-based UI, error checking, and transaction-based interactions. This improves Vagrant’s resilience to bad changes. The update will also resolve issues around Vagrant’s index file
growing too big
or
being too fragile
.
While the existing Ruby plugin API will continue to be supported, Vagrant will support a new plugin API with the help of the
HashiCorp go-plugin library
. Plugin authors will be able to create Vagrant plugins in Go or any other language with GRPC support. A plugin SDK will provide common utilities and helpers for plugin development.
You will be able to port plugins written with the new API across platforms and release them independently of Vagrant’s releases. A compiled Go plugin's cross-platform capability will ease distribution and usage by simplifying current requirements for complex dependency management.
A long-standing issue with some plugins and third-party distributions packaging Vagrant revolves around Vagrant shipping its own Ruby. This causes friction when you want to
run Vagrant on the latest versions of Ruby
or when Ruby plugins depend on packages that
require specific libraries
. Moving away from bundling Ruby with Vagrant will give users more control over the Ruby runtime. For example, systems such as Arch or Fedora that distribute their own Vagrant package can safely depend on system Ruby.
We have developed a number of Go libraries for other HashiCorp products and other infrastructure tooling by various vendors and communities. The move to Go will allow Vagrant to more fully join this community, gaining the ability to use (and contribute to) tools that already exist, as well as lean on the Go expertise that exists at Hashicorp.
HashiCorp tools such as
go-plugin
and
glint
helped us explore the possibilities around writing Vagrant in Go while maintaining backward compatibility. Moving forward, other Go libraries will allow Vagrant to accommodate a wide variety of new plugins. For example, you can use Go bindings for different virtualization software to build new plugins for many more backends.
Vagrant has more than 70 built-in plugins. This proliferation has introduced new development constraints. Currently, any fix that may be required on a built-in plugin is dependent on the entirety of Vagrant before it can be released. With Vagrant 3.0, as we continue to improve the plugin API and portability, we can extract plugins from Vagrant so you can update them faster and easier.
The focus will mainly be on provider and provisioner plugins. Once these plugins are externalized, you will be able to upgrade provider and provisioner plugins independent of Vagrant. Command plugins will remain internal and still depend on Vagrant’s release cycle.
We recognize that porting a project to another language can be disruptive, so we aim to make it as seamless as possible. We will not deprecate any interfaces or features you require as a Vagrant end user or plugin developer during this process. First, we will port the core of Vagrant to Go. Once that is stable, we will work on built-in plugins. This iterative process ensures full compatibility between the Go and Ruby Vagrant runtimes, a requirement for our own built-in plugins to remain functional.
We will spread these changes over two minor releases and one major release:
The Vagrant 2.3 release will include the initial alpha version of the Go-based Vagrant implementation. It will be shipped in the Vagrant installer packages and made available as an optional executable. You will be able to choose to use the new implementation, but you may experience some instability. Throughout the Vagrant 2.3 release cycle, we will work to stabilize the Go-based Vagrant implementation and begin to port built-in plugins. The
vagrant
command will continue to run the traditional Ruby-based Vagrant implementation.
The Vagrant 2.4 release will feature the new Go implementation of Vagrant as the primary executable. It will include a migration command, which will import existing data from the Vagrant home and project directories. Vagrantfiles will also support HCL and data structures.
