-
Notifications
You must be signed in to change notification settings - Fork 16
Home
Welcome to the Midden wiki. The goal of this wiki is to provide information about Midden, its architecture, and guides for its use. At the moment, this is a monumental task and is very much a work in progress.
If you are looking into using Midden then the best thing to do is to post to Discussions or to contact Bryan Carlson to start a conversation. Going it alone at this point is not recommended.
But for those who like a little pain, here ya go:
Midden makes some assumptions about how your data are managed. It also requires an individual who has some technical expertise and is familiar with things like git, Github, and command line tools. Before moving on to steps 2 and beyond, it may be a good idea to review what we consider best practices in data management and see if your current (or future) data system is compatible. You also may want to review some of the technical details of Midden
As explain in the technical overview, one component of Midden is a web tool. The web tool consists of an online data catalog and an online metadata editor. As Midden is not a hosted service, the web tools need to be setup/installed for your organization.
But you are in luck! There are a lot of free hosting services out there where you can deploy your very own instance of Midden. So far our users have been successful with Github Pages, Azure Static Web Apps, and a on-prem server.
A guide for installing the web tools using Github Pages can be found here.
When first deployed, the Midden web tools are configured with demo information. Your organization do not want that. And you do not want that. So lets change it!
The Midden web tools reference two user-generated files; app-config.json and catalog.json. The first, app-config.json, contains things like the name of your organization and pre-configured tags, spatial information (like your research fields), and data quality flags. The second, catalog.json, contains all of your metadata.
Details of updating the app-config.json file using Github Pages can be found here.
We will discuss updating the catalog.json file in later steps.
Hey, you have done pretty well so far! You have your very own instance of Midden running on a server somewhere and have it customized for your organization. Now lets use this thing for what it was meant for: describing data! To do that you will want to use the Midden Editor to describe your data, then download a .midden file (which is just a .json file) that contains your metadata, then save it along-side the dataset it describes.
For more details on using the Midden Editor, reference this page here.
The MiddenCli is a command line interface that is (or will be) used for all sorts of admin-y tasks. Primarily, it is used to crawl through all of your organization's data stores (where the data are stored) and aggregate all of the metadata. MiddenCli is cross-platform so can be used on Windows, Mac, and Linux.
You can download the latest OS-specific version from our releases page.
A guide to using MiddenCli is here.
Remember that second user-defined file we mentioned so long ago (Step 3)? You already took care of the first (app-config.json), now it is time to tackle catalog.json. You generate the file using the collate command in MiddenCli.
More information about running collate to generate a catalog.json file can be found here.
Once you have a catalog.json file, you need to update the demo version of it in your instance of the Midden web tools. This can be done a lot of different ways depending on where you deployed your instance of the web tools and how you configured your app-config.json.
For a guide on updating catalog.json within Github Pages, see here.
Well look at fancy you. You did all of the steps! Now you just need to keep things updated. It's not too terrible. You basically do Step 6 every so often (or automate it... which has been done) to update Midden Catalog as new metadata are added to your data stores. For a breakdown though, it's basically this:
- Someone adds a dataset to a data store
- Someone uses the organization's Midden Editor to create metadata
- Someone saves the metadata file along-side the newly added dataset
- Someone/something runs
MiddenCli collateto generate acatalog.jsonfile - Someone/something updates the
catalog.jsonfile that is referenced by the organization's Midden Catalog
That's it!
Actually, there is one more thing. You may want to keep your version of Midden up to date. New features and bug-fixes are constantly being addressed.
For a guide on updating Midden see here.
For a guide on updating Midden hosted by Github Pages, see here.