Tyba Forecast API Walkthrough

Welcome to this walkthrough on the Tyba Forecast API.

By the end of this guide, you’ll know how to retrieve the most recent energy and ancillary price forecasts, as well as how to retrieve older vintages of forecasts.

Prerequisites

Ensure you have the tyba_client python packages installed.

pip install tyba-client

Setting Up

Firstly, we need to set up our environment and create a Tyba client instance. For that, make sure you’ve set the TYBA_PAT in your environment.

After instantiating a Tyba client, one can access the forecast client by calling client.forecast

from datetime import datetime, time
import pytz
import os
import pandas as pd
from tyba_client.client import Client

PAT = os.environ["TYBA_PAT"]
client = Client(PAT)
forecast = client.forecast

Fetching Energy Price Forecasts

For energy price forecasts (DA and RT), we recommend using the probabilistic endpoints which provide more detailed information about forecast uncertainty.

Probabilistic Forecast Endpoints

The Tyba Forecast API supports fetching probabilistic forecasts for energy prices. These endpoints allow users to request forecasts with specified quantiles, providing insights into the variability and confidence of the forecasted values.

For all probabilistic endpoints, the minimum quantile one can request is 0.10 and the maximum is 0.9. Quantiles requested outside of the range will result in a 400 status code.

Fetching Most Recent Probabilistic Forecasts

Let’s suppose we’re interested in fetching probabilistic day-ahead and real-time energy price forecasts for a node from 10/10/23 until 10/17/23.

node_name = "HB_HOUSTON"
tz = pytz.timezone("US/Central") # Can be any timezone but results will always be returned in the node-local timezone
start_time = tz.localize(datetime(2023, 10, 10)) # localized datetime is required to avoid ambiguity
end_time = tz.localize(datetime(2023, 10, 17)) # localized datetime is required to avoid ambiguity

Day-Ahead Forecasts

For day-ahead forecasts, use the most_recent_probabilistic method without additional parameters:

# Fetching probabilistic day-ahead forecasts
quantiles = [0.10, 0.50, 0.90]
da_forecasts = forecast.most_recent_probabilistic("HB_HOUSTON", "da", start_time, end_time, quantiles)

Real-Time Forecasts

For real-time forecasts, especially if you need subhourly data, use the most_recent_probabilistic method with additional parameters:

Use product=”rt” to fetch the SPP, and product=”rt_lmp” to fetch the LMP price. You may only have access to one of these models, so reach out to type support for questions.

# Fetching probabilistic real-time forecasts with subhourly data
rt_forecasts = forecast.most_recent_probabilistic(
    object_name="HB_HOUSTON",
    product="rt",
    start_time=start_time,
    end_time=end_time,
    quantiles=[0.10, 0.50, 0.90],
    forecast_type="rolling-24hr",
    predictions_per_hour=4,        # Default for subhourly RT forecasts
    prediction_lead_time_mins=15,  # Default for subhourly RT forecasts, pass `0` for rt_lmp
    horizon_mins=1515              # Default for subhourly RT forecasts
)

Note: The parameters predictions_per_hour, prediction_lead_time_mins, and horizon_mins depend on the model you have access to. The values shown above are the recommended defaults for real-time forecasts.

Understanding the Returned Data

The most_recent_probabilistic endpoint returns a list of forecast dictionaries with the following structure:

{
  'quantiles': [0.1, 0.5, 0.9],
  'values': [14.60, 16.42, 18.93],
  'mean_value': 16.55,
  'datetime': '2023-10-10T00:00:00-05:00',
  'forecasted_at': '2023-10-09T09:00:00-05:00',
  'forecast_type': 'day-ahead'
}

Each element represents:

quantiles: The requested probability levels
values: The corresponding forecasted values for each quantile
mean_value: The mean expected value for the price
datetime: The date and time the forecast corresponds to (timezone-aware)
forecasted_at: When the forecast was made (timezone-aware)
forecast_type: The type of forecast model used

Processing Probabilistic Forecasts

Here’s an example of how to process probabilistic forecasts into a more usable format:

def process(df):
    return (
        df.assign(datetime=lambda d: pd.to_datetime(d["datetime"]))
        .explode(["quantiles", "values"])
        .drop(["forecasted_at", "forecast_type"], axis=1)
        .pivot(index="datetime", columns="quantiles", values="values")
        .assign(mean=lambda d: d.apply(lambda x: df.loc[df["datetime"] == x.name, "mean_value"].iloc[0], axis=1))
        .tz_localize(None)
    )

This would convert the data into a pivot table like:

+----------------------+-----------+-----------+-----------+-----------+
|  quantiles           | 0.1       | 0.5       | 0.9       | mean      |
+----------------------+-----------+-----------+-----------+-----------+
|  datetime            |           |           |           |           |
+======================+===========+===========+===========+===========+
| 2023-10-10 00:00:00  | 14.603392 | 16.423639 | 18.932876 | 16.55000  |
+----------------------+-----------+-----------+-----------+-----------+
| 2023-10-10 01:00:00  | 14.126083 | 15.910089 | 17.825208 | 15.97123  |
+----------------------+-----------+-----------+-----------+-----------+

Working with Vintaged Forecasts

Similar to point forecasts, you can also retrieve vintaged probabilistic forecasts.

Fetching Vintaged Probabilistic Forecasts

To fetch day-ahead probabilistic forecasts made at a specific time before the forecast period:

# Fetching vintaged probabilistic day-ahead forecasts made a day prior, before 10am
da_vintaged_forecasts = forecast.vintaged_probabilistic(
    "HB_HOUSTON",
    "da",
    start_time,
    end_time,
    quantiles=[0.10, 0.50, 0.90],
    days_ago=1,
    before_time=time(10, 0)
)

For real-time probabilistic forecasts with subhourly data:

# Fetching vintaged probabilistic real-time forecasts with subhourly data
rt_vintaged_forecasts = forecast.vintaged_probabilistic(
    object_name="HB_HOUSTON",
    product="rt",
    start_time=start_time,
    end_time=end_time,
    quantiles=[0.10, 0.50, 0.90],
    days_ago=1,
    before_time=time(10, 0),
    forecast_type="rolling-24hr",
    predictions_per_hour=4,
    prediction_lead_time_mins=15,
    horizon_mins=1515
)

Fetching Forecasts by Vintage

You may also fetch probabilistic forecasts based on when they were created:

# Fetching day-ahead probabilistic forecasts by vintage time
vintage_start_time = tz.localize(datetime(2023, 10, 9))
vintage_end_time = tz.localize(datetime(2023, 10, 10))

da_by_vintage = forecast.by_vintage_probabilistic(
    "HB_HOUSTON",
    "da",
    [0.10, 0.50, 0.90],
    vintage_start_time,
    vintage_end_time
)

# Fetching real-time probabilistic forecasts by vintage time with subhourly data
rt_by_vintage = forecast.by_vintage_probabilistic(
    object_name="HB_HOUSTON",
    product="rt",
    quantiles=[0.10, 0.50, 0.90],
    vintage_start_time=vintage_start_time,
    vintage_end_time=vintage_end_time,
    forecast_type="rolling-24hr",
    predictions_per_hour=4,
    prediction_lead_time_mins=15,
    horizon_mins=1515
)

Working with Ancillary Prices

In contrast to energy prices, ancillary prices should be fetched using the point forecast endpoints (non-probabilistic).

In ERCOT, Ancillary prices are system-wide. This means that to fetch them, you’d pass "ERCOT" as the object name. Here’s how to fetch forecasts for reg_up and reg_down:

# Using point forecast endpoints for ancillary prices
reg_up_forecasts = forecast.most_recent("ERCOT", "reg_up", start_time, end_time)
reg_down_forecasts = forecast.most_recent("ERCOT", "reg_down", start_time, end_time)

Each forecast dictionary will be structured as:

{
  "value": float,                    # The forecasted value
  "datetime": string (ISO 8601),     # The date and time the forecast is corresponding to (timezone-aware)
  "forecasted_at": string (ISO 8601), # The date and time when the forecast was made (timezone-aware)
  "forecast_type": string  #  either "day-ahead" or "rolling-24hr" if the forecast was made by the day-ahead forecast model or the rolling 24hr real-time model
}

For vintaged ancillary forecasts, use the corresponding point forecast methods:

# Fetching vintaged ancillary forecasts a day prior, before 10am
reg_up_vintaged = forecast.vintaged("ERCOT", "reg_up", start_time, end_time, days_ago=1, before_time=time(10, 0))

# Fetching ancillary forecasts by vintage time
reg_down_by_vintage = forecast.by_vintage("ERCOT", "reg_down", vintage_start_time, vintage_end_time)

With the above steps, you’ve successfully learned how to fetch various types of forecasts using the Tyba client. Remember to use probabilistic endpoints for energy prices (DA/RT) and point endpoints for ancillary prices. Feel free to reach out with any questions!

Additional Information on Subhourly Forecasts

Note: Subhourly forecasts are only available for the real-time product (“rt”) and require specifying forecast_type = "rolling-24hr".

The parameters for subhourly forecasts are:

predictions_per_hour (int): The number of predictions made per hour (e.g., 4 for 15-minute intervals).
prediction_lead_time_mins (int): The lead time for the predictions in minutes (e.g., 15 for predictions made 15 minutes before the forecast time).
horizon_mins (int): The horizon in minutes for which the forecasts are made (typically 1515).

These parameters can be used in combination with existing methods for fetching both point and probabilistic forecasts, as shown in the examples above.