Tyba Forecast API Walkthrough

Welcome to this walkthrough on the Tyba Forecast API.

By the end of this guide, you’ll know how to retrieve the most recent energy and ancillary price forecasts, as well as how to retrieve older vintages of forecasts.

Prerequisites:

Ensure you have the tyba_client python packages installed.

pip install tyba-client

Setting Up:

Firstly, we need to set up our environment and create a Tyba client instance. For that, make sure you’ve set the TYBA_PAT in your environment.

After instantiating a Tyba client, one can access the forecast client by calling client.forecast

from datetime import datetime, time
import pytz
import os
import pandas as pd
from tyba_client.client import Client

PAT = os.environ["TYBA_PAT"]
client = Client(PAT)
forecast = client.forecast

Fetching Most Recent Energy Price Forecasts:

Let’s suppose we’re interested in HB_HOUSTON’s day-ahead and energy prices from 10/10/23 until 10/17/23.

node_name = "HB_HOUSTON"
tz = pytz.timezone("US/Central") # Can be any timezone but results will always be returned in the node-local timezone
start_time = tz.localize(datetime(2023, 10, 10)) # localized datetime is required to avoid ambiguity
end_time = tz.localize(datetime(2023, 10, 17)) # localized datetime is required to avoid ambiguity

In order to fetch the most recent prices for any hour interval within the start-time / end-time range, we can use the forecast.most_recent method.

It takes 4 required parameters and 1 optional parameter:

• object_name (str): The name of the object we are fetching prices for. In our case, this is “HB_HOUSTON”

• product (str): The name of the product whose prices we want. For us, this will be either “da” for day-ahead or “rt” for real-time.

• start_time (datetime.datetime): The beginning of the datetime range to fetch prices for (inclusive)

• end_time (datetime.datetime): The end of the datetime range to fetch prices for (exclusive)

• forecast_type (Optional[str]): Either None, “day-ahead”, or “rolling-24hr”. If “day-ahead”, filters to forecasts from the day-ahead model. “rolling-24hr” will filter to the real-time model. None will perform no filtering (the default)

da_forecasts = forecast.most_recent("HB_HOUSTON", "da", start_time, end_time)
rt_forecasts = forecast.most_recent("HB_HOUSTON", "rt", start_time, end_time)

forecast.most_recent will return a list of forecast dictionaries.

Each forecast dictionary will be structured as so:

{
  "value": float,                    # The forecasted value
  "datetime": string (ISO 8601),     # The date and time the forecast is corresponding to (timezone-aware)
  "forecasted_at": string (ISO 8601), # The date and time when the forecast was made (timezone-aware)
  "forecast_type": string  #  either "day-ahead" or "rolling-24hr" if the forecast was made by the day-ahead forecast model or the rolling 24hr real-time model
}

For example, da_forecasts[0] may look like:

{
  'value': 17.12,
  'datetime': '2023-10-25T00:00:00-05:00',
  'forecasted_at': '2023-10-24T06:00:00-05:00'
}

Understanding Vintaged Forecasts:

While the most recent forecasts provide insights into Tyba’s latest predictions, there may be times you want to look at older vintages for backtesting.

This is where vintaged forecasts come in handy.

For example, to retrieve the RT forecasts made by the day-ahead bid close of the previous day, you can utilize the forecast.vintaged endpoint.

This endpoint takes in the same parameters as forecast.most_recent above, but includes two extra parameters:

• days_ago (int): The number of days prior relative to the datetime hour of the forecast that the forecast should have been made by.

• before_time (datetime.time): The time of day at which the forecast should have been made before on the day specified by days_ago

• exact_vintage (bool): If true, will only pull forecasts made at exactly days_ago and before_time. If False, will pull forecasts made at or before.

forecast_type can also be used in vintaged for filtering to a specific forecast type.

The day-ahead bid-close happens at 10am on the previous day. Thus, we can pass

days_ago=1 and before_time=time(10,0)

Here is a code snippet:

# Fetching forecasts made a day prior, before 10am
rt_vintaged_forecasts = forecast.vintaged("HB_HOUSTON", "rt", start_time, end_time, days_ago=1, before_time=time(10, 0))

The return format is the exact same as in forecast.most_recent

For example, rt_vintaged_forecasts[0] may look like:

{
  'value': 15.29,
  'datetime': '2023-10-10T00:00:00-05:00',
  'forecasted_at': '2023-10-09T09:00:00-05:00'
}

As we can see above, the forecasted_at suggests the forecast was made at 9am the day prior of the datetime value.

Fetching forecasts by the vintage

Users may be interested in fetching forecasts based on their vintage time rather than the hour the forecast was made for. forecast.by_vintage can be used to fetch forecasts from forecast vintages generated between vintage_start_time and vintage_end_time.

It takes 4 required parameters and 1 optional parameter:

• object_name (str): The name of the object we are fetching prices for. In our case, this is “HB_HOUSTON”

• product (str): The name of the product whose prices we want. For us, this will be either “da” for day-ahead or “rt” for real-time.

• vintage_start_time (datetime.datetime): The beginning of the datetime range of the vintages to fetch prices for (inclusive)

• vintage_end_time (datetime.datetime): The end of the datetime range of the vintages to fetch prices for (exclusive)

• forecast_type (Optional[str]): Either None, “day-ahead”, or “rolling-24hr”. If “day-ahead”, filters to forecasts from the day-ahead model. “rolling-24hr” will filter to the real-time model. None will perform no filtering (the default)

The returned data will look the same as the return format in forecast.most_recent.

Working with Ancillaries:

In ERCOT, Ancillary prices are system-wide. This means that to fetch them, you’d pass "ERCOT" as the object name. Here’s how to fetch forecasts for reg_up and reg_down:

reg_up_forecasts = forecast.most_recent("ERCOT", "reg_up", start_time, end_time)
reg_down_forecasts = forecast.most_recent("ERCOT", "reg_down", start_time, end_time)

---

With the above steps, you’ve successfully fetched various types of forecasts using the Tyba client. Feel free to reach out with any questions!

Probabilistic Forecast Endpoints

The Tyba Forecast API also supports fetching probabilistic forecasts. These endpoints allow users to request forecasts with specified quantiles, providing insights into the variability and confidence of the forecasted values.

For all probabilistic endpoints, the minimum quantile one can request is 0.10 and the maximum is 0.9. Quantiles requested outside of the range will result in a 400 status code.

Fetching Probabilistic Forecasts for the Most Recent Vintages

The forecast.most_recent_probabilistic method retrieves the most recent probabilistic forecasts for a given node, product, and time range, considering specified quantiles.

Parameters:

• object_name (str): The name of the object for which prices are being fetched.

• product (str): The product type, such as “da” for day-ahead or “rt” for real-time.

• start_time (datetime.datetime): The start of the datetime range (must be localized with timezone offset).

• end_time (datetime.datetime): The end of the datetime range (must be localized with timezone offset).

• quantiles (list[float]): A list of quantiles to fetch, e.g., [0.10, 0.50, 0.90].

• forecast_type (Optional[str]): Filters to either “day-ahead”, “rolling-24hr”, or None for no filtering.

Example:

quantiles = [0.10, 0.50, 0.90]
da_forecasts = forecast.most_recent_probabilistic("HB_HOUSTON", "da", start_time, end_time, quantiles)

Fetching Vintaged Probabilistic Forecasts

The forecast.vintaged_probabilistic method retrieves probabilistic forecasts made prior to a specified time.

Parameters:

• Includes all parameters from forecast.most_recent_probabilistic.

• days_ago (int): Specifies how many days prior the forecast was made.

• before_time (datetime.time): The time by which the forecast was made on the specified day.

• exact_vintage (bool): If true, fetches forecasts made exactly at the specified time.

Example:

da_vintaged_forecasts = forecast.vintaged_probabilistic("HB_HOUSTON", "da", start_time, end_time, quantiles, days_ago=1, before_time=time(10, 0))

Fetching Probabilistic Forecasts by Vintage

The forecast.by_vintage_probabilistic method allows fetching probabilistic forecasts based on the vintage time.

Parameters:

• Includes all parameters from forecast.most_recent_probabilistic.

• vintage_start_time (datetime.datetime): The start of the datetime range for the vintage.

• vintage_end_time (datetime.datetime): The end of the datetime range for the vintage.

Example:

rt_vintage_forecasts = forecast.by_vintage_probabilistic("HB_HOUSTON", "rt", quantiles, vintage_start_time, vintage_end_time, forecast_type="day-ahead")

Processing Probabilistic Forecasts

Probabilistic forecasts return a structure with quantiles and corresponding values. Below is an example of how you can process these forecasts.

Returned Structure:

{
  'quantiles': [0.1, 0.5, 0.9],
  'values': [14.60, 16.42, 18.93],
  'datetime': '2024-01-01T00:00:00-06:00',
  'forecasted_at': '2023-12-31T06:00:00-06:00',
  'forecast_type': 'day-ahead'
}

Processing Example:

def process(df):
    return (
        df.assign(datetime=lambda d: pd.to_datetime(d["datetime"]))
        .explode(["quantiles", "values"])
        .drop(["forecasted_at", "forecast_type"], axis=1)
        .pivot(index="datetime", columns="quantiles", values="values")
        .tz_localize(None)
    )

=>

quantiles	0.1	0.5	0.9
datetime
2024-01-01 00:00:00	14.603392	16.423639	18.932876
2024-01-01 01:00:00	14.126083	15.910089	17.825208
2024-01-01 02:00:00	14.244211	15.741732	17.914553
2024-01-01 03:00:00	15.510665	16.847187	19.105508
2024-01-01 04:00:00	16.127306	17.808273	20.297680
…	…	…	…

Fetching Subhourly Forecasts

The Tyba Forecast API supports fetching subhourly forecasts. These endpoints allow users to request forecasts with higher frequency within each hour, providing more granular insights.

Note: Subhourly forecasts are only available for the real-time product (“rt”) and require specifying forecast_type = “rolling-24hr”.

For subhourly forecasts, you can specify the number of predictions per hour (predictions_per_hour) and the prediction lead time (prediction_lead_time_mins). This allows you to fetch forecasts that are made multiple times within each hour.

Parameters for Subhourly Forecasts

The following parameters are used for fetching subhourly forecasts:

• predictions_per_hour (int): The number of predictions made per hour (e.g., 4 for 15-minute intervals).

• prediction_lead_time_mins (int): The lead time for the predictions in minutes (e.g., 75 for predictions made 75 minutes before the forecast time).

• horizon_mins (int): The horizon in minutes for which the forecasts are made.

• forecast_type (str): Must be set to “rolling-24hr” for subhourly forecasts.

These parameters can be used in combination with existing methods for fetching forecasts.

Fetching Most Recent Subhourly Forecasts

To fetch the most recent subhourly forecasts, use the most_recent method with the additional parameters for subhourly forecasting.

Example:

from datetime import datetime
import pytz
import os
from tyba_client.client import Client

PAT = os.environ["TYBA_PAT"]
client = Client(PAT)
forecast = client.forecast

node_name = "HB_HOUSTON"
tz = pytz.timezone("US/Central")
start_time = tz.localize(datetime(2023, 10, 10))
end_time = tz.localize(datetime(2023, 10, 17))

# Fetching subhourly forecasts with 15-minute intervals and a lead time of 15 minutes
subhourly_forecasts = forecast.most_recent(
    object_name=node_name,
    product="rt",
    start_time=start_time,
    end_time=end_time,
    forecast_type="rolling-24hr",
    predictions_per_hour=4,
)

Fetching Vintaged Subhourly Forecasts

To fetch vintaged subhourly forecasts, use the vintaged method with the additional parameters for subhourly forecasting.

Example:

from datetime import datetime, time
import pytz
import os
from tyba_client.client import Client

PAT = os.environ["TYBA_PAT"]
client = Client(PAT)
forecast = client.forecast

node_name = "HB_HOUSTON"
tz = pytz.timezone("US/Central")
start_time = tz.localize(datetime(2023, 10, 10))
end_time = tz.localize(datetime(2023, 10, 17))

# Fetching vintaged subhourly forecasts made a day prior before 10am with 15-minute intervals and a lead time of 15 minutes
vintaged_subhourly_forecasts = forecast.vintaged(
    object_name=node_name,
    product="rt",
    start_time=start_time,
    end_time=end_time,
    forecast_type="rolling-24hr",
    days_ago=1,
    before_time=time(10, 0),
    predictions_per_hour=4,
    prediction_lead_time_mins=75
)

Fetching Subhourly Forecasts by Vintage

To fetch subhourly forecasts by their vintage time, use the by_vintage method with the additional parameters for subhourly forecasting.

Example:

from datetime import datetime
import pytz
import os
from tyba_client.client import Client

PAT = os.environ["TYBA_PAT"]
client = Client(PAT)
forecast = client.forecast

node_name = "HB_HOUSTON"
tz = pytz.timezone("US/Central")
vintage_start_time = tz.localize(datetime(2023, 10, 10))
vintage_end_time = tz.localize(datetime(2023, 10, 17))

# Fetching subhourly forecasts by their vintage time with 15-minute intervals and a lead time of 15 minutes
subhourly_forecasts_by_vintage = forecast.by_vintage(
    object_name=node_name,
    product="rt",
    vintage_start_time=vintage_start_time,
    vintage_end_time=vintage_end_time,
    forecast_type="rolling-24hr",
    predictions_per_hour=4,
    prediction_lead_time_mins=75
)

Fetching Subhourly Probabilistic Forecasts

To fetch subhourly probabilistic forecasts, use the most_recent_probabilistic, vintaged_probabilistic, or by_vintage_probabilistic methods with the additional parameters for subhourly forecasting.

Example for Most Recent Probabilistic Subhourly Forecasts:

from datetime import datetime
import pytz
import os
from tyba_client.client import Client

PAT = os.environ["TYBA_PAT"]
client = Client(PAT)
forecast = client.forecast

node_name = "HB_HOUSTON"
tz = pytz.timezone("US/Central")
start_time = tz.localize(datetime(2023, 10, 10))
end_time = tz.localize(datetime(2023, 10, 17))
quantiles = [0.10, 0.50, 0.90]

probabilistic_subhourly_forecasts = forecast.most_recent_probabilistic(
    object_name=node_name,
    product="rt",
    start_time=start_time,
    end_time=end_time,
    forecast_type="rolling-24hr",
    quantiles=quantiles,
    predictions_per_hour=4,
    prediction_lead_time_mins=75
)