This wiki has been closed because there have been no edits or logs made within the last 60 days. This wiki is now eligible for being adopted. To adopt this wiki please go to Requests for adoption and make a request. If this wiki is not adopted within 6 months it may be deleted. Note: If you are a bureaucrat on this wiki you can go to Special:ManageWiki and uncheck the "closed" box to reopen it.

Tools

From Open Data Scotland
Revision as of 16:21, 12 October 2020 by AshMcC (talk | contribs) (→‎Data Formats)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Data Formats

Common data formats are .csv, .json and .txt files.

Common, more exact, formats include: Turtle (.ttl) and CSV on the Web (.csvw).

Best Practice

For spreadsheets read Data Organization in Spreadsheets by Karl W. Broman & Kara H. Woo https://www.tandfonline.com/doi/full/10.1080/00031305.2017.1375989

Tools

Reading Data

Flatten Tool (json to csv) https://flatten-tool.readthedocs.io/en/latest/examples/

Google Sheets: App Script https://developers.google.com/apps-script and Google Query Language https://developers.google.com/chart/interactive/docs/querylanguage. And a library for version controlling: Clasp https://github.com/google/clasp

Microsoft Excel

Power Query is an Extract, Transform and Load (ETL) tool built in to Microsoft Excel (from 2016) and Microsoft's Power BI. It has a user interface which allows users to transform data without needing to code, although coding is possible in the "M" language. https://docs.microsoft.com/en-us/power-query/power-query-what-is-power-query

Mistakes and open/closed methodologies: Austerity spreadsheet mistake https://www.theatlantic.com/business/archive/2013/04/forget-excel-this-was-reinhart-and-rogoffs-biggest-mistake/275088/

Data Visualisation

Microsoft PowerBI

Tableau

R Shiny

D3.js