Update on July 23, 2019: Since the time I originally made this post, Debian has moved from version 9.6 Stretch to 10.0 Buster. I’ve updated the post accordingly. Thankfully, nothing changed but the name of the Vagrant box to use! If you still want to use Stretch, replace everywhere that it says stretch64 with buster64. Some of your error messages may be different, though…
- Add this new Vagrant Box into Vagrant where ‘mynewbox’ is any name you choose (preferrably similar to the name of the box you just added) and ‘mynew.box’ is the file name of the box you just created $ vagrant box add mynewbox mynew.box.
- Create box This is how to install something in a VM and export it as a Vagrant box and use it locally. First copy the Vagrantfile from below and change the box to the box you want as a base. Run vagrant up to create the Virtual Machine and vagrant ssh to login.
- This is some of the initial work we’ve done moving to Windows VMs with Vagrant. Windows VM Vagrant’s documentation for Windows guest VMs is pretty good. We’re in the process of evaluating Windows 10 Enterprise LTSC (the OS formerly known as Windows 10 “IoT Enterprise”/LTSB) which is comparable to Windows 10 Enterprise version 1809/RS5.
- # goto default directory $ cd VirtualBox VMs/ # create base box from VM $ vagrant package -base Win10x64 -output Win10x64.box # add box $ vagrant box add lupin/windows10 Win10x64.box # check vagrant boxes $ vagrant box list Create and run test project.
In the project, set up process after creating the Vagrant file we need to specify a box. To add a box we use the vagrant box add command and specify the box name. HashiCorp hosts a box catalog here where you can search and download boxes. For example to add the xenial 64 bit box we use the command below. Vagrant box add ubuntu/xenial64.
Lately, for my Linux development, I’ve been using Vagrant, using VirtualBox as a provider. In Vagrant, I set up a Debian box and do my development on that. It’s nice and easy to set up, and if I want to restart with a clean installation of Debian, it’s very easy to do.
I’ll write three posts about how I set it up. In this first post, I detail the steps that I use to create a basic Debian box, with no GUI. In my second post, I describe the steps I use to set up VirtualBox’s Guest Additions, so that I can share files between my host Windows system and my Debian Vagrant box. In the third post, I give the steps that I use to set up a GUI.
If you know what you’re doing, and you want to skip all the explanations and just see the steps involved, feel free to look at my summary!
Debian’s Vagrant Box
The Debian project provides standard Vagrant boxes in the Vagrant Cloud box catalog. You can read about them on the Debian wiki, and look at the provided boxes on https://app.vagrantup.com/debian/. The latest version of Debian is version 10, codenamed Buster. At the time of writing this post, the latest point release was 10.0, and I’m using the 64-bit version of Debian. So, the Vagrant box I’m using is debian/buster64.
The Debian Vagrant box provides a standard basic Debian system, containing all packages with priority required, important, and standard. Note that it does NOT include a GUI; you’re just going to get a command-line version. We’ll look at how to set up a GUI in the third post.
Prerequisites
To set this up, you need to have both VirtualBox and Vagrant installed. You can download VirtualBox from the website https://www.virtualbox.org. The download page is https://www.virtualbox.org/wiki/Downloads, and the installation instructions are in Chapter 2 of the User Manual. You can download Vagrant from the website https://www.vagrantup.com, where the download page is at https://www.vagrantup.com/downloads.html, and the installation instructions are at https://www.vagrantup.com/intro/getting-started/install.html. I installed VirtualBox first, then Vagrant. I don’t know if the order matters, but it made sense to me, and I know that works!
Creating the Vagrantfile
Once you’ve got Vagrant and VirtualBox installed, you’re going to want to create the configuration file for your box. This file is called Vagrantfile. I’ve created a folder under my home directory called vagrant-VMs, and underneath that, another folder called buster. I’ve created multiple folders under vagrant-VMs, one for each Vagrant configuration, but in this post, I’m only discussing my Buster installation.
Here’s looking at my folder in File Explorer:
Open a command prompt, then change to the vagrant-VMsbuster folder:
Next, use Vagrant to create the Vagrantfile for us:
This will create the Vagrantfile that configures the usage of the Vagrant box. The box won’t actually be downloaded until you try to bring the box up for the first time.
Here’s a screenshot.
You might want to take a look at the Vagrantfile; it’s just a text file (actually, a Ruby program file), so you can look at it in your favourite text editor, such as Notepad, Notepad++, or Vim. There’s actually very little in the file; most of it is commented-out configuration, giving examples of things you may want to configure. But, for now, we’ll just leave things as they are.
Here’s a screenshot of the complete file (click to view it larger):
Creating the Virtual Machine
The next step is to create and start up the virtual machine. We’ll use the “vagrant up” command to do that. The first time you run this, it will connect to the Hashicorp atlas (a repository of Vagrant boxes, maintained by the developers of Vagrant) and download a copy of the box (called the “base box”). Then, it will create a virtual machine as a copy of the downloaded base box; subsequent executions will just use the virtual machine you’ve already created.
Note: Vagrant creates a folder to hold all its configuration data under your home directory, with the name .vagrant.d. If you’re curious where the downloaded base box goes, it goes underneath that folder, in a subfolder named boxes. On Windows, I use File History to back up my files; because these boxes are so big, and because I can easily re-download them, I don’t want to back them up, so I configure File History to not backup the .vagrant.dboxes folder. There’s another folder that I also don’t back up, but I’ll mention that next…
Second note: the virtual machine gets created in a folder in your home directory called “VirtualBox VMs”. Once again, I don’t want to back these up, so I add this folder to my list of excluded folders in File History. You may want to back them up, but as I said before, you can easily recreate these VMs, so it’s a waste of backup space to back them up.
OK, so let’s create our Virtual Machine:
You’ll see a whole bunch of messages. Vagrant goes through the following steps when it creates the VM:
- Imports the debian/buster64 base box and uses it to create a new VM
- Configures the networking of the new VM
- Boots the VM, and waits for the booting to complete
- Uses SSH to connect to the VM, and replaces the SSH key in the VM (more on that later)
- Checks to see if the VirtualBox guest additions are installed.
You’ll now see one of two messages about the VirtualBox guest additions. If they are not installed in the Vagrant box (they shouldn’t be, but version 10.0.0 of the box seems to include them), you’ll get a scary error message telling you that you should install them, and that a lot of things won’t work until you install them. Ignore that for now; I’ll look at installing the guest additions in my next post. If they are installed (version 10.0.0, I’m looking at you!), you may get a slightly less scary message that the guest additions on the VM don’t match the installed version of VirtualBox. Don’t worry about that, as we’ll install the proper version in the next post.
At the time of writing, I also get a VERY scary error message about attempting to rsync a shared folder. Once again, don’t worry about that; we’ll resolve that in the next post as well.
With regards to SSH: the buster64 box that was downloaded from the Hashicorp atlas has a well-known SSH key, but you really want that to be secret. But, if it was secret, we wouldn’t be able to connect to the VM. So, when we first bring up the VM, Vagrant connects to the VM using the well-known key, then generates a secret key, and replaces the well-known key with our new secret key. Finally, it disconnects and reconnects using the new key, just to make sure everything is working OK.
Here’s a screenshot of all the output (click to enlarge):
You’ll notice that it doesn’t appear that the VM is running. The “vagrant up” command completes and returns us to the command prompt, and there isn’t a VirtualBox GUI. But, don’t worry, VirtualBox is running; you just can’t see it! If you really want, you can start up Windows’ Task Manager, and see the VBoxHeadless process listed under Background Processes. Or, you can start up VirtualBox, and you’ll see that you’ve got a running VM!
Bringing Debian Up-to-date
There may be security patches issued by Debian that aren’t incorporated into your imported box. The next thing we’re going to do is to bring all the software up-to-date.
Log in to the VM using the “vagrant ssh” command:
This will connect to the VM using SSH, and will log you in to an account named “vagrant”. This account was present in the Vagrant box you downloaded; it’s a standard convention used by Vagrant boxes. If you ever need to log in without using the “vagrant ssh” command, the password is also “vagrant”.
Here’s a screenshot:
Once you’re logged in, you’ll be at the Debian command prompt. We’ll want to execute the following commands:
The first command will connect to the Debian package repository, and download a list of all of the latest versions of Debian packages. The second command will download and install any packages that have been updated in the repository since the Vagrant base box was created.
Here’s a screenshot of executing the update:
Vagrant For Windows 10
and another of executing the upgrade (click to enlarge). Note that your packages being updated almost certainly will be different from mine.
Depending on which packages have been updated, there are two other possible commands you may need to execute. If you get a message about some packages being held back, execute this command:
And, if you get a message that some packages are no longer needed, you can remove them with:
Actually, there’s really no harm in executing both of these commands anyways, since we’re setting up a new VM; they just might not do anything.
And now, we’re done! We’ve got an up-to-date Debian system running.
Log out of the SSH session to return to the Windows command prompt:
Shutting down the Virtual Machine
The last thing to do is to shut down the virtual machine. That’s very easy to do from the Windows command prompt. Execute:
And you’ll get a message that Vagrant is gracefully shutting down the virtual machine.
Next time you want to start the VM back up, just go to your vagrant-VMsbuster folder (or whatever you called it), and re-execute “vagrant up”. It won’t need to recreate the VM; it’ll just use it as it was the last time you used it.
Do you want to clean up?
For completeness, I should mention how you can clean up what you’ve done.
If you want to delete the VM that you created, execute:
That will delete the virtual machine. You can easily recreate the VM, though, just be executing “vagrant up” again. This is a nice easy way to start your VM over fresh if you mess it up somehow. It leaves your Vagrantfile intact, so any configuration changes you made in there will be used the next time you recreate the virtual machine. But, any changes you made inside the VM (such as updating Debian!) will need to be redone.
If you also want to remove the base box, execute:
If you want to use it again, you’ll need to re-download it; executing “vagrant up” will automatically do that for you if you need it.
Next steps
It’s very useful to have the VirtualBox guest additions installed on your system; I detail how to do that in my next post. This will allow you to share files between your VM and your Windows machine that the VM is running on. Also, if you’re running a GUI (which I’ll describe how to set up in my third post), it allows you to resize the window, which is very handy!
Last updated: 2020-03-05 :: Published: 2015-01-25 :: [ history ][UPDATE 2020/03/05]: While Vagrant served me well for years, I do not recommend using it for local development anymore and advise to use Docker instead, for which I wrote an entire tutorial series which I invite you to read.
This article shows how to deal with Windows' specificities while trying to work with Vagrant. If you are not familiar at all with the latter, I suggest you go through this Vagrant tutorial first.
Note: the following was tested on Windows 8, but the steps described below shouldn't change too much between the different versions.
Vagrant ssh
The first issue I came across was that vagrant ssh doesn't work out of the box. I was greeted with a constructive message:
Fine. Let's install Git, then (considering it is not already the case).
Git install
The key is when the 'Adjusting your PATH environment' screen pops up:
You want to select 'Use Git and optional Unix tools from the Windows Command Prompt'.
Now I know the message in red looks quite scary, but honestly, unless you are a hardcore user of the Windows console, there is not much to worry about. Basically it will override some of the commands and add a few others.
Personally, it never caused me any trouble.
If you are still a bit worried tho, be reassured: none of this is irreversible. All you would need to do is uninstall Git, or update the PATH variable removing the incriminated part.
More on that in a minute.
Try to vagrant ssh your VM again, this time it should do it (you might need to open a new terminal for the update to take effect, tho).
What if Git is installed already?
Well, it was the case for me as well.
You could always remove it and install it again, but there is another way.
You will have to do manually what the installation of Git could have done for you, but fortunately it is quite trivial:
- Open the Control Panel
- Go to System and Security
- Click on System, then on the Change Settings button
- Display the Advanced tab and click on Environment Variables...
- Look for the Path variable in the System variables list, select it then Edit...
At the end of the string, add the path to Git's bin (something like 'C:Program FilesGitbin') (don't forget to add a semicolon first to separate it from the previous path):
Validate and close the different menus. Try to vagrant ssh your box again, it should work (again, you might need to open a new terminal first).
You probably guessed it already, but if you don't want Git's commands to override the Windows ones anymore, all you need to do is to remove that bit.
You will need to find another way for ssh to work though!
Ah, but wait. There is another way.
PuTTY
Remember that error message we initially got trying to ssh the box? Let's have a look at the second part of it:
I wasn't joking when I said it was constructive, because it really tells you what to do.
The Windows console works ok but let's be honest, in the long run it is a real pain to use. It does the trick for a quick vagrant ssh but when the time comes to actually do some work on an Ubuntu server for example, a better shell is desirable.
Enter PuTTY
PuTTY is a very lightweight tool that allows to do a lot of cool stuff. Some of you are probably familiar with it already, and using it jointly with Vagrant is quite nice.
Vagrant Add Box
We will use it to ssh our boxes, and rely on the info given by the message above to that purpose.
First, download it if that is not the case already (the first putty.exe link will do).
Download puttygen.exe as well, we are going to need it.
PuTTY and PuTTYGen are stand-alone applications (no need to install them), so just double click on the .exe files.
Let's open PuTTYGen first: PuTTY uses its own key format, and we need to convert Vagrant's one first. Click on File then Load private key, select the file indicated by the error message earlier (e.g. 'C:/path/to/project/.vagrant/machines/default/virtualbox/private_key').
Once selected, PuTTY is kind enough to tell us what to do with it:
Ensure SSH-2 RSA is selected, and that the number in Number of bits in a generated key is 2048. Then click on Save private key (don't set a passphrase) and save it under your own user directory's .ssh folder, as 'vagrant_private_key'. From now on, we will use this key for all the Vagrant boxes.
Close PuTTYGen and open PuTTY. In the Hostname field, type 127.0.0.1. In the Port one, 2222. Ensure SSH is selected and, in the Saved Sessions field, type vagrant and click Save:
Go to Connection then Data, and in the Auto-login username field, enter 'vagrant':
Next, still under Connection, go to SSH then Auth. Browse for the key you generated earlier in the Private key file for authentication field. Now head back to the Session menu, save again the 'vagrant' one.
Now click on Open: if everything went alright, you should now be in your Vagrant box ＼(^o^)／
Using multiple Vagrant boxes simultaneously
Now let's say you already have a box running, and you need to start a second one. You vagrant up it, the Virtual Machine boots and you want to SSH it. But all the boxes cannot use the same SSH port!
It all happens when the box is being booted:
See the highlighted line? Seeing port 2222 was busy already, Vagrant picked the port 2200 instead.
Now to SSH it using PuTTY, open it, load the 'vagrant' session, and in the Port field, replace '2222' with '2200'. Click Open: there you are, connected to the second box.
Known limitations
Shared folders and symlinks
One of the fairly known limitations of using Vagrant on Windows with VirtualBox is that the latter won't let you create symlinks on the shared folders for security reasons. This quickly becomes problematic when dealing with npm packages, for example.
One of the workarounds is to use the 'no bin link' parameter (e.g. npm install --no-bin-link), but this is not always enough.
Fortunately, there is a way to bypass this restriction. In your Vagrantfile, add the following piece of config:
As Windows won't let standard users create symlinks, you now need to start your Vagrant box in administrator mode (open a Windows terminal in admin mode before running vagrant up, for example). Make sure no other box is already running though, as it won't start if VirtualBox is already running in standard mode.
Maximum path length
Another recurring problem comes from the fact that Windows' maximum length for a path is 255 characters. Again, this is quickly an issue when dealing with npm packages, especially when they have dependencies, themselves having dependencies, etc.
The solution in that case is to create a symbolic link between the 'node_modules' directory and another directory outside of the shared folders.
Which brings us to our practical example.
Practical example: npm packages
So you have this project relying on npm packages. You tried to install them using --no-bin-link but no luck, looks like some of the paths are too long.
Fear not, Macless: update your Vagrant config as shown above to allow the creation of symlinks, boot your VM in admin mode, create a destination directory for your npm packages somewhere outside of the shared folder and create a symlink between it and the 'node_modules' one:
Et voilà.
Note: this implies preventing the 'node_modules' directory from being versionned.
Conclusion
Vagrant Box File
Here you go, now using Vagrant on Windows in decent conditions.
The process can look a bit convoluted, and really it is. It took me quite a while to put everything together, and if today I am rather satisfied, I am still a bit bugged about the multiple Vagrant boxes part. Having to check the SSH port and update it in the PuTTY session everytime is a bit annoying, even though dealing with several instances at the same time might be an edge case.
Anyways, if you have any suggestions about that, don't hesitate to leave a comment.
Comments