5-minute Guide to Packaging

Norman Walsh
Last updated 2012-01-05

Packaging, part of the suite of Configuration Management features introduced in MarkLogic 5, makes it easy to move application server and database configurations between machines.

Do either of these situations sound familiar to you?

  • You've been working on an application on your development machine. Now it's time to move your application to the staging or testing servers. What follows is a tedious process of reviewing the settings on your development machine and applying them to the staging machine. How sure are you that you got all the indexes just right?

  • You've got a certified configuration that you want to deploy onto a new cluster. Getting the hardware setup and installing the server itself isn't too hard, but now you have to make sure that all the application servers and databases are setup. Can you see another tedious process coming?

The packaging support in MarkLogic 5 will make these tasks much less painful. Packaging allows you to take a snapshot of the current settings of one or more application servers and databases. That snapshot is saved as an XML document. If you move that XML document onto another machine, you can directly apply those settings on the new machine. You can be completely sure that all of the settings are the same, without any of the tedium!

Let's take it for a test drive!

The trouble with testing a feature like packaging is that it's really designed to help you keep two different setups in sync. That means at least two instances of MarkLogic, each running the same (or at least an overlapping) set of databases and appservers. That's a lot of setup for one little tutorial.

So we're going to cheat just a little. We're going to assume you only have one machine running MarkLogic 5. We're going to call that the “local machine”. We're going to talk about a hypothetical other machine, called the “target machine”, but you don't really need one of those to do this tutorial.

Here's the scenario: you've been building an application on your local machine that uses the Documents database. Now it's time to deploy that application on the target machine. There are a bunch of things you have to do to complete that deployment. Packaging makes one of them a lot easier: copying the settings for the Documents database.

Start by navigating to http://host:8002/manage/v1/list/package. This will take you to the Figure 1, “Package export” screen. You'll need to replace host with the address of the local machine where MarkLogic 5 is installed. That's often localhost, but it could be something else. (If you are running MarkLogic on your local machine, then you can go straight there by clicking this link: http://localhost:8002/manage/v1/list/package.)

Figure 1. Package export

On the package export screen, select the Documents database. In practice, you'd select all of the application servers and databases associated with your application, but to keep things simple, we'll just use the Documents database. Provide a filename (or just leave package.xml, if you prefer) and click “Download Package” to get your package.

The package document is XML. You aren't expected to open it, but there's no reason not to. It'll look something like this:

Figure 2. Package XML

Do not make changes to this file. In order to function as a package, it must remain schema valid and satisfy other constraints. It is entirely possible to build XML applications that integrate packaging into other systems by generating or modifying package XML files, but it's not something to approach casually.

Back to our tutorial. After you've downloaded the package, this is the place in the real world where you'd take the package XML file and transfer it to the target machine. Often that just means sending it in email or putting it on a shared file system, but if you don't have network access to the target machine, you could burn it onto a CD and physically move it to the target machine.

For the purpose of this tutorial, we're going to pretend you have two machines by making some changes on the one machine you do have and then pretending its the target machine. We're going to make a few random changes to the settings of the Documents database; don't do this on a system that you care about. Or backup the database before you start and restore it afterwards. Obviously, this is something you should only do if you have complete control over the MarkLogic server; if you're sharing this machine with others, you should coordinate this experiment with them.

Here's what to do: open the Admin UI at http://host:8001/. (Click here if running locally: http://localhost:8001) If you're not familiar with the Admin UI, the initial page will look something like this:

Figure 3. Admin UI

Choose the Documents database:

Figure 4. Documents Database Settings

Make some changes to it: change “stemmed searches” and “word positions”:

Figure 5. Adjusted Documents Database Settings

Change some of the “trailing wildcard” settings. Then click the "OK" button to confirm your changes. Finally, add an element range index using the "Element Range Indexes" link in the left-hand menu. The details of what you enter here don't matter.

Now the machine that used to be your local machine is your target machine. We can use packaging to restore the Documents database back to the original configuration, just exactly as if it was coming from some other machine.

Navigate to http://host:8002/manage/v1/package/compare (or click here if running locally: http://localhost:8002/manage/v1/package/compare). This will take you to the Figure 6, “Package upload” screen.

Figure 6. Package upload

Browse or type the name of the package file and click “Upload”.

The server will now compare the settings in the package that you've uploaded with the current settings on the target machine. The results are displayed on the Figure 7, “Package differences” screen. Your results won't look exactly like these, of course, but you'll see some changed settings, some new settings, and possibly some deleted settings, as we've shown here.

Figure 7. Package differences

Figure 8, “More package differences” shows that the element range index you recently added will now be deleted if you import these settings.

Figure 8. More package differences

The package differences screen shows what's new, deleted, and changed. What you'll see is the effect of undoing all the changes you made in the Admin UI a few minutes ago. In a real world scenario, this would be your opportunity to review the changes and decide if you were willing to accept them all. There's no provision in 5.0-1 for applying only selected changes. If you aren't happy with some of the settings, you'll have to go back to the local machine, adjust them, and rebuild the package.

When you are happy with the new settings, click “Apply” and you'll get to the Figure 9, “Package installed” screen.

Figure 9. Package installed

The resulting ticket contains a full history of everything that was changed by the install process. You can revert this ticket, if you need to back out the change. There are also REST and XQuery APIs to this functionality, so you can script access to the packaging functionality.

Packaging support in MarkLogic 5 has some limitations. We'll be working to extend the functionality in future releases.

Packaging only copies database and application server settings. You are still responsible for managing the code associated with application servers and the content of databases.

Comments

  • Even despite its current limitations, pretty awesome already. Really filling a large gap. Looking forward to future extension..