Tyba Forecast API Walkthrough

Welcome to this walkthrough on the Tyba Forecast API.

By the end of this guide, you’ll know how to retrieve the most recent energy and ancillary price forecasts, as well as how to retrieve older vintages of forecasts.

Prerequisites:

Ensure you have the tyba_client python packages installed.

pip install tyba-client

Setting Up:

Firstly, we need to set up our environment and create a Tyba client instance. For that, make sure you’ve set the TYBA_PAT in your environment.

After instantiating a Tyba client, one can access the forecast client by calling client.forecast

from datetime import datetime, time
import pytz
import os
import pandas as pd
from tyba_client.client import Client

PAT = os.environ["TYBA_PAT"]
client = Client(PAT)
forecast = client.forecast

Fetching Most Recent Energy Price Forecasts:

Let’s suppose we’re interested in HB_HOUSTON’s day-ahead and energy prices from 10/10/23 until 10/17/23.

node_name = "HB_HOUSTON"
tz = pytz.timezone("US/Central") # Can be any timezone but results will always be returned in the node-local timezone
start_time = tz.localize(datetime(2023, 10, 10)) # localized datetime is required to avoid ambiguity
end_time = tz.localize(datetime(2023, 10, 17)) # localized datetime is required to avoid ambiguity

In order to fetch the most recent prices for any hour interval within the start-time / end-time range, we can use the forecast.most_recent method.

It takes 4 required parameters and 1 optional parameter:

  • object_name (str): The name of the object we are fetching prices for. In our case, this is “HB_HOUSTON”

  • product (str): The name of the product whose prices we want. For us, this will be either “da” for day-ahead or “rt” for real-time.

  • start_time (datetime.datetime): The beginning of the datetime range to fetch prices for (inclusive)

  • end_time (datetime.datetime): The end of the datetime range to fetch prices for (exclusive)

  • forecast_type (Optional[str]): Either None, “day-ahead”, or “rolling-24hr”. If “day-ahead”, filters to forecasts from the day-ahead model. “rolling-24hr” will filter to the real-time model. None will perform no filtering (the default)

da_forecasts = forecast.most_recent("HB_HOUSTON", "da", start_time, end_time)
rt_forecasts = forecast.most_recent("HB_HOUSTON", "rt", start_time, end_time)

forecast.most_recent will return a list of forecast dictionaries.

Each forecast dictionary will be structured as so:

{
  "value": float,                    # The forecasted value
  "datetime": string (ISO 8601),     # The date and time the forecast is corresponding to (timezone-aware)
  "forecasted_at": string (ISO 8601), # The date and time when the forecast was made (timezone-aware)
  "forecast_type": string  #  either "day-ahead" or "rolling-24hr" if the forecast was made by the day-ahead forecast model or the rolling 24hr real-time model
}

For example, da_forecasts[0] may look like:

{
  'value': 17.12,
  'datetime': '2023-10-25T00:00:00-05:00',
  'forecasted_at': '2023-10-24T06:00:00-05:00'
}

Understanding Vintaged Forecasts:

While the most recent forecasts provide insights into Tyba’s latest predictions, there may be times you want to look at older vintages for backtesting.

This is where vintaged forecasts come in handy.

For example, to retrieve the RT forecasts made by the day-ahead bid close of the previous day, you can utilize the forecast.vintaged endpoint.

This endpoint takes in the same parameters as forecast.most_recent above, but includes two extra parameters:

  • days_ago (int): The number of days prior relative to the datetime hour of the forecast that the forecast should have been made by.

  • before_time (datetime.time): The time of day at which the forecast should have been made before on the day specified by days_ago

  • exact_vintage (bool): If true, will only pull forecasts made at exactly days_ago and before_time. If False, will pull forecasts made at or before.

forecast_type can also be used in vintaged for filtering to a specific forecast type.

The day-ahead bid-close happens at 10am on the previous day. Thus, we can pass

days_ago=1 and before_time=time(10,0)

Here is a code snippet:

# Fetching forecasts made a day prior, before 10am
rt_vintaged_forecasts = forecast.vintaged("HB_HOUSTON", "rt", start_time, end_time, days_ago=1, before_time=time(10, 0))

The return format is the exact same as in forecast.most_recent

For example, rt_vintaged_forecasts[0] may look like:

{
  'value': 15.29,
  'datetime': '2023-10-10T00:00:00-05:00',
  'forecasted_at': '2023-10-09T09:00:00-05:00'
}

As we can see above, the forecasted_at suggests the forecast was made at 9am the day prior of the datetime value.

Fetching forecasts by the vintage

Users may be interested in fetching forecasts based on their vintage time rather than the hour the forecast was made for. forecast.by_vintage can be used to fetch forecasts from forecast vintages generated between vintage_start_time and vintage_end_time.

It takes 4 required parameters and 1 optional parameter: * object_name (str): The name of the object we are fetching prices for. In our case, this is “HB_HOUSTON” * product (str): The name of the product whose prices we want. For us, this will be either “da” for day-ahead or “rt” for real-time. * vintage_start_time (datetime.datetime): The beginning of the datetime range of the vintages to fetch prices for (inclusive) * vintage_end_time (datetime.datetime): The end of the datetime range of the vintages to fetch prices for (exclusive) * forecast_type (Optional[str]): Either None, “day-ahead”, or “rolling-24hr”. If “day-ahead”, filters to forecasts from the day-ahead model. “rolling-24hr” will filter to the real-time model. None will perform no filtering (the default)

The returned data will look the same as the return format in forecast.most_recent.

Working with Ancillaries:

In ERCOT, Ancillary prices are system-wide. This means that to fetch them, you’d pass "ERCOT" as the object name. Here’s how to fetch forecasts for reg_up and reg_down:

reg_up_forecasts = forecast.most_recent("ERCOT", "reg_up", start_time, end_time)
reg_down_forecasts = forecast.most_recent("ERCOT", "reg_down", start_time, end_time)

With the above steps, you’ve successfully fetched various types of forecasts using the Tyba client. Feel free to reach out with any questions!

Probabilistic Forecast Endpoints

The Tyba Forecast API also supports fetching probabilistic forecasts. These endpoints allow users to request forecasts with specified quantiles, providing insights into the variability and confidence of the forecasted values.

For all probabilistic endpoints, the minimum quantile one can request is 0.10 and the maximum is 0.9. Quantiles requested outside of the range will result in a 400 status code.

Fetching Probabilistic Forecasts for the Most Recent Vintages

The forecast.most_recent_probabilistic method retrieves the most recent probabilistic forecasts for a given node, product, and time range, considering specified quantiles.

Parameters:

  • object_name (str): The name of the object for which prices are being fetched.

  • product (str): The product type, such as “da” for day-ahead or “rt” for real-time.

  • start_time (datetime.datetime): The start of the datetime range (must be localized with timezone offset).

  • end_time (datetime.datetime): The end of the datetime range (must be localized with timezone offset).

  • quantiles (list[float]): A list of quantiles to fetch, e.g., [0.10, 0.50, 0.90].

  • forecast_type (Optional[str]): Filters to either “day-ahead”, “rolling-24hr”, or None for no filtering.

Example:

quantiles = [0.10, 0.50, 0.90]
da_forecasts = forecast.most_recent_probabilistic("HB_HOUSTON", "da", start_time, end_time, quantiles)

Fetching Vintaged Probabilistic Forecasts

The forecast.vintaged_probabilistic method retrieves probabilistic forecasts made prior to a specified time.

Parameters:

  • Includes all parameters from forecast.most_recent_probabilistic.

  • days_ago (int): Specifies how many days prior the forecast was made.

  • before_time (datetime.time): The time by which the forecast was made on the specified day.

  • exact_vintage (bool): If true, fetches forecasts made exactly at the specified time.

Example:

da_vintaged_forecasts = forecast.vintaged_probabilistic("HB_HOUSTON", "da", start_time, end_time, quantiles, days_ago=1, before_time=time(10, 0))

Fetching Probabilistic Forecasts by Vintage

The forecast.by_vintage_probabilistic method allows fetching probabilistic forecasts based on the vintage time.

Parameters:

  • Includes all parameters from forecast.most_recent_probabilistic.

  • vintage_start_time (datetime.datetime): The start of the datetime range for the vintage.

  • vintage_end_time (datetime.datetime): The end of the datetime range for the vintage.

Example:

rt_vintage_forecasts = forecast.by_vintage_probabilistic("HB_HOUSTON", "rt", quantiles, vintage_start_time, vintage_end_time, forecast_type="day-ahead")

Processing Probabilistic Forecasts

Probabilistic forecasts return a structure with quantiles and corresponding values. Below is an example of how you can process these forecasts.

Returned Structure:

{
  'quantiles': [0.1, 0.5, 0.9],
  'values': [14.60, 16.42, 18.93],
  'datetime': '2024-01-01T00:00:00-06:00',
  'forecasted_at': '2023-12-31T06:00:00-06:00',
  'forecast_type': 'day-ahead'
}

Processing Example:

def process(df):
    return (
        df.assign(datetime=lambda d: pd.to_datetime(d["datetime"]))
        .explode(["quantiles", "values"])
        .drop(["forecasted_at", "forecast_type"], axis=1)
        .pivot(index="datetime", columns="quantiles", values="values")
        .tz_localize(None)
    )

=>

quantiles

0.1

0.5

0.9

datetime

2024-01-01 00:00:00

14.603392

16.423639

18.932876

2024-01-01 01:00:00

14.126083

15.910089

17.825208

2024-01-01 02:00:00

14.244211

15.741732

17.914553

2024-01-01 03:00:00

15.510665

16.847187

19.105508

2024-01-01 04:00:00

16.127306

17.808273

20.297680

…

…

…

…