Product Update: March

This month we are announcing a stack of features and improvements that make running our platform locally as a developer, on-premise, or privately in your own cloud even easier.  These improvements have been spurred on by our customers, partners, and community who use our technology to: 

• Aggregate data into private ESG data hubs – Learn more…
• Power seamless data collection & sharing – Learn more…
• Automate Meltano jobs and manage configuration – Learn more…

TL;DR

• Custom Auth0 identity providers
• Datasets preview in the Matatika Lab
• User interface, API and scheduler for Meltano projects
• Developer insights from GitHub
• Improved spreadsheets data source

Custom Auth0 identity providers

When running our default Community Edition configuration, or in the Matatika Cloud, we manage all the user authentication with the fabulous Auth0 identity service.  This means all users get enterprise grade security out of the box.  Now, developers from our customers and community are able to configure their own instance of Auth0 to ensure all organisation identity management is within their IT management.  Learn more…

Datasets preview in the Matatika Lab

Users of our platform have published over 65K datasets to their members who login to our App workspaces.  Generally these datasets are created by users who also aggregate data with pipelines in our Lab.  Our Community Edition users pointed out how useful it would be to see a preview of the data immediately from within the Lab.  Of course we listened to that feedback and have now released a new feature to see a preview of charts and tables all from within the Lab.  Learn more…

User interface, API, and scheduler for Meltano

At Matatika, we have been contributing to the Meltano open source extract and load technology for several years.  In that time, the Meltano team’s focus has moved increasingly toward the needs of data engineers – users who are more technical and comfortable managing code and infrastructure.  Consequently, they recently deprecated their UI and API recently.  We recognised that the “no code” Matatika Lab and API would provide a UI and integration bridge for companies that have focused their data teams attention on data analytics rather than engineering.  Now we have a standalone platform for hosting Meltano projects – the Matatika Community Edition; we have released a minimal version of our Lab as a UI and scheduler utility for any Meltano project.  Learn more…

Developer insights from GitHub 

We think development teams should have great data too!  Our users are already collecting data from Auth0, Trello, Google, and spreadsheets.  Now, with our instant insights for GitHub, development teams can rapidly setup a data warehouse to understand their team output using commit and issue data from GitHub.  Learn more…

Improved spreadsheets data source

Spreadsheets are some of the most common and easiest ways to capture and move data around.  This month we have improved several aspects of how we import spreadsheet data using our platform.  Making it easier to aggregate and share data is always our priority.  Learn more…

Fixes and upgrades

• Upgraded to meltano~=2.13.0
• Improved support for running and converting Jupyter notebooks in pipelines
• Fixed a bug in workspace deployment where multiple environments with multiple components with commands would not deploy
• Fixed a bug where we were persisting array kind settings to our database incorrectly and sometimes losing information
• Fixed a bug with updating plugins with commands that had a container spec
• Improved start / stop of docker-compose, preventing named volume conflicts
• Ownership of workspace files retained/set as current user
• Hidden plugins can now be deployed when using fully qualified plugin name
• Workspace deployment with entities now deployed in dependent order
• Remove workspace configuration(s) for workspace when deleted from MATATIKA_WORKSPACES_HOME
• Added environment for enabling/disabling search
• Fixed a bug parsing fully-qualified plugin names containing multiple occurrences of the variant delimiter string (–) correctly

Finally, if any of the above has piqued your interest, please consider giving us a star on GitHub!