Waterflow

Waterflow is Australia’s first and only independent and free-to-use large-scale aggregator of water trading data for the Murray Darling basin and Australia. An initiative of Marsden Jacob, and funded through the Australian government’s BRII initiative, Waterflow gives water users clarity on all key aspects of the market; including pricing, dam storage levels, trade limits and available allocations.

The Challenge

The Waterflow Project presented three significant challenges:

• Data Collection

  Water trading data is typically a few weeks old. We needed up to date data which meant going direct to Australian government data, state water registries, brokers and other intermediaries. We currently automatically acquire data from 44 sources, each in a different format.

• Data Cleaning

  Water trading data is often misleading either due to “off market” trades or human error, so it needs to be expertly cleaned to improve market transparency and save time for water users when undertaking their research.

• Usability

  Water trading is complex, with a range of geographic and legislative restrictions on what trades are allowed between zones. We needed simple interfaces to help farmers understand their trading options. In addition, our farmers are often in remote locations on smartphones with slow, expensive data.

Our Approach

The data collection challenge was solved by developing a multi-phase data pipeline allowing systematic scraping, importing and cleaning of trade data. This pipeline approach also allowed us to quickly add new suppliers.

Data cleaning was tackled in two ways. First, we use statistical models to exclude extreme outliers. We then present a tailored interface to allow a Marsden Jacob expert to manually remove any remaining questionable data in just a few minutes per day.

The usability of the platform was a key focus from the outset. Our User Experience (UX) specialist worked closely with Marsden Jacob staff and conducted a series of user interviews. This feedback informed all aspects of the build, including the development of two “low data” native mobile apps for remote farmers.

The Results

Waterflow.io was launched to broad praise by the industry, with over 1000 water market participants now using the platform. And, we regularly receive requests from brokers volunteering their data to the platform, consolidating its position as the nation’s largest aggregator of both real time and historic water market data. The applications have also been recognised by the ACCC for contributing to greater industry transparency.