Node Search Price Pull Example
Notebook Overview
This notebook illustrates how to use the “node search” API endpoint. This endpoint allows users to search by location, node name, iso, or a combination. When searching by location, a list of the nearest nodes is returned. Once the user has identified the node they want to use, they can then pull historical prices for this node using the “get prices” API endpoint.
Imports
First, let’s import software packages, the Client API and your PAT.
[1]:
import time
import os
import pandas as pd
from tyba_client.client import Client, Market
PAT = os.environ["TYBA_PAT"]
client = Client(PAT)
# Relevant API services
services = client.services
lmp = services.lmp
ancillary = services.ancillary
Node Search Functions
Next, let’s use the lmp.search_nodes function to search for a node. There are multiple ways a user can search for a node. To confirm successful search, status_code = 200. The function will return the node location, node ISO, node name, node ID, and node distance to the search location (if specified).
1. Search by location / address
Let’s search by location and extract the list of nodes into a dataframe.
[2]:
res = lmp.search_nodes(location="dallas, texas")
res.raise_for_status()
df = pd.DataFrame(res.json()["nodes"])
[3]:
df
[3]:
| node/iso | node/name | node/id | node/distance-meters | |
|---|---|---|---|---|
| 0 | ERCOT | EMPCT_RN | 10018267018 | 8198.673431 |
| 1 | ERCOT | HB_NORTH | 10000697078 | 12354.211404 |
| 2 | ERCOT | LZ_NORTH | 10000698392 | 12354.211404 |
| 3 | ERCOT | PRCRK_RN | 10018267016 | 12448.495097 |
| 4 | ERCOT | MCSES_UNIT6 | 10000698679 | 14209.794577 |
| 5 | ERCOT | MCSES_UNIT7 | 10000698680 | 14209.794577 |
| 6 | ERCOT | MCSES_UNIT8 | 10000698681 | 14209.794577 |
| 7 | ERCOT | LIGSW_RN | 10018248462 | 21842.460551 |
| 8 | ERCOT | LHSES_UNIT1 | 10000698659 | 24281.578207 |
| 9 | ERCOT | LHSES_UNIT2 | 10000698660 | 24281.578207 |
| 10 | ERCOT | FRNYPP_1_CCU | 10000698581 | 28839.581651 |
| 11 | ERCOT | FRNYPP_2_CCU | 10000698582 | 28839.581651 |
| 12 | ERCOT | FRNYPP_CC1 | 10000698343 | 28839.581651 |
| 13 | ERCOT | FRNYPP_CC2 | 10000698344 | 28839.581651 |
| 14 | ERCOT | CNTRY_RN | 10017419312 | 28907.071045 |
| 15 | ERCOT | ANOL_ESS_RN | 10018287496 | 34680.822268 |
| 16 | ERCOT | HLSES_UNIT3 | 10000698608 | 39688.083699 |
| 17 | ERCOT | HLSES_UNIT4 | 10000698609 | 39688.083699 |
| 18 | ERCOT | HLSES_UNIT5 | 10000698610 | 39688.083699 |
| 19 | ERCOT | ELS_RN_1 | 10016962333 | 42838.585540 |
| 20 | ERCOT | LILY_RN | 10016498291 | 43822.820037 |
| 21 | ERCOT | DSWL_BES1 | 10018287497 | 44573.640963 |
| 22 | ERCOT | NOBLESLR_ALL | 10016874016 | 44906.180346 |
| 23 | ERCOT | MDANP_CT1_2 | 10000698682 | 45619.085651 |
| 24 | ERCOT | MDANP_CT3_4 | 10000698683 | 45619.085651 |
We can also search by lat/lng or street address.
[4]:
res = lmp.search_nodes(location="29.760427, -95.369804")
res = lmp.search_nodes(location="3 NRG Pkwy, Houston, TX 77054")
2. Search by portion of node name
[5]:
res = lmp.search_nodes(node_name_filter="HB_")
res.raise_for_status()
df = pd.DataFrame(res.json()["nodes"])
[6]:
df
[6]:
| node/id | node/name | node/iso | |
|---|---|---|---|
| 0 | 10000698380 | HB_BUSAVG | ERCOT |
| 1 | 10000697077 | HB_HOUSTON | ERCOT |
| 2 | 10000698382 | HB_HUBAVG | ERCOT |
| 3 | 10000697078 | HB_NORTH | ERCOT |
| 4 | 10015999590 | HB_PAN | ERCOT |
| 5 | 10000697079 | HB_SOUTH | ERCOT |
| 6 | 10000697080 | HB_WEST | ERCOT |
| 7 | 10016973502 | IHB_7_IRONHORSE_BATGNODE | CAISO |
| 8 | 10004176539 | PGHB_6_PDRP01-APND | CAISO |
| 9 | 10004074430 | PGHB_6_PDRP02-APND | CAISO |
| 10 | 10002701851 | PGHB_6_PDRP03-APND | CAISO |
| 11 | 10004262229 | PGHB_6_PDRP04-APND | CAISO |
| 12 | 10015847591 | PGHB_6_PDRP05-APND | CAISO |
| 13 | 10015853877 | PGHB_6_PDRP06-APND | CAISO |
| 14 | 10015880548 | PGHB_6_PDRP07-APND | CAISO |
| 15 | 10016481077 | PGHB_6_PDRP08-APND | CAISO |
| 16 | 10016481089 | PGHB_6_PDRP09-APND | CAISO |
| 17 | 10016019849 | PGHB_6_PDRP10-APND | CAISO |
| 18 | 10016240024 | PGHB_6_PDRP11-APND | CAISO |
| 19 | 10016240072 | PGHB_6_PDRP12-APND | CAISO |
| 20 | 10016481091 | PGHB_6_PDRP14-APND | CAISO |
| 21 | 10016481092 | PGHB_6_PDRP15-APND | CAISO |
| 22 | 10016481078 | PGHB_6_PDRP16-APND | CAISO |
| 23 | 10016495007 | PGHB_6_PDRP17-APND | CAISO |
| 24 | 10016656810 | PGHB_6_PDRP18-APND | CAISO |
3. Search by portion of node name and location / address
We can combine multiple search filters to narrow down further.
[7]:
res = lmp.search_nodes(node_name_filter="HB_", location="Houston, Texas")
4. Search for a location but with an ISO override to constrain search to the specified ISO
We can even search for a location but then specify an ISO within which to look for nodes.
[8]:
res = lmp.search_nodes(location="toledo, OH", iso_override="MISO")
Pull Energy Prices
Now that we’ve searched for the node(s) in question, let’s extract the relevant node directory information.
[9]:
df
[9]:
| node/id | node/name | node/iso | |
|---|---|---|---|
| 0 | 10000698380 | HB_BUSAVG | ERCOT |
| 1 | 10000697077 | HB_HOUSTON | ERCOT |
| 2 | 10000698382 | HB_HUBAVG | ERCOT |
| 3 | 10000697078 | HB_NORTH | ERCOT |
| 4 | 10015999590 | HB_PAN | ERCOT |
| 5 | 10000697079 | HB_SOUTH | ERCOT |
| 6 | 10000697080 | HB_WEST | ERCOT |
| 7 | 10016973502 | IHB_7_IRONHORSE_BATGNODE | CAISO |
| 8 | 10004176539 | PGHB_6_PDRP01-APND | CAISO |
| 9 | 10004074430 | PGHB_6_PDRP02-APND | CAISO |
| 10 | 10002701851 | PGHB_6_PDRP03-APND | CAISO |
| 11 | 10004262229 | PGHB_6_PDRP04-APND | CAISO |
| 12 | 10015847591 | PGHB_6_PDRP05-APND | CAISO |
| 13 | 10015853877 | PGHB_6_PDRP06-APND | CAISO |
| 14 | 10015880548 | PGHB_6_PDRP07-APND | CAISO |
| 15 | 10016481077 | PGHB_6_PDRP08-APND | CAISO |
| 16 | 10016481089 | PGHB_6_PDRP09-APND | CAISO |
| 17 | 10016019849 | PGHB_6_PDRP10-APND | CAISO |
| 18 | 10016240024 | PGHB_6_PDRP11-APND | CAISO |
| 19 | 10016240072 | PGHB_6_PDRP12-APND | CAISO |
| 20 | 10016481091 | PGHB_6_PDRP14-APND | CAISO |
| 21 | 10016481092 | PGHB_6_PDRP15-APND | CAISO |
| 22 | 10016481078 | PGHB_6_PDRP16-APND | CAISO |
| 23 | 10016495007 | PGHB_6_PDRP17-APND | CAISO |
| 24 | 10016656810 | PGHB_6_PDRP18-APND | CAISO |
Pull all ISO nodes
Let’s first pull the ERCOT ISO directory. We’ll filter this directory to use in any energy price pulls.
[10]:
services.get_all_isos().json()
[10]:
['PJMISO', 'CAISO', 'NEISO', 'NYISO', 'ERCOT', 'SPPISO', 'MISO']
[11]:
price_nodes = lmp.get_all_nodes(iso='ERCOT').json()
node_directory = pd.DataFrame(price_nodes)
[12]:
node_directory
[12]:
| da_start_year | rt_end_year | zone | name | type | substation | da_end_year | id | rt_start_year | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 2011.0 | 2025 | SOUTH | OXY_OXY_CCG1 | GENERATOR | OXYCHEM CC | 2025.0 | 10000698721 | 2011 |
| 1 | 2022.0 | 2025 | SOUTH | SLCN_ESS_ALL | GENERATOR | NaN | 2025.0 | 10016671726 | 2022 |
| 2 | 2024.0 | 2025 | SOUTH | ANEM_ESS_RN | GENERATOR | NaN | 2025.0 | 10017318833 | 2024 |
| 3 | 2011.0 | 2025 | NORTH | WHCCS_CC1 | GENERATOR | WOLF HOLLOW GEN | 2025.0 | 10000698379 | 2011 |
| 4 | 2011.0 | 2025 | HOUSTON | SCLP_PUN4 | GENERATOR | SWEENY COGEN | 2025.0 | 10000698780 | 2011 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 1071 | 2022.0 | 2025 | WEST | QUEEN_BA_RN | GENERATOR | NaN | 2025.0 | 10016874015 | 2022 |
| 1072 | 2024.0 | 2025 | HOUSTON | RBN_BESS1 | LOAD | NaN | 2025.0 | 10017290064 | 2024 |
| 1073 | 2018.0 | 2025 | SOUTH | SANTACRU_ALL | GENERATOR | SANTACRU | 2025.0 | 10004183883 | 2018 |
| 1074 | 2023.0 | 2025 | WEST | PALE_ESS_EN | GENERATOR | NaN | 2025.0 | 10017177379 | 2023 |
| 1075 | 2011.0 | 2025 | HOUSTON | WAP_WAP_G2 | GENERATOR | WA PARISH | 2025.0 | 10000698870 | 2011 |
1076 rows × 9 columns
Helper Functions
This takes the raw API payload and converts it into a dataframe with a datetime index:
[13]:
def reformat_df(result):
df = pd.concat({node:pd.DataFrame(results['prices'],index=pd.to_datetime(results['datetimes'])) for node,results in result.items()},axis=1)
return df.droplevel(axis=1,level=1).tz_localize(None)
This is a function to simplify pulling 1 or more nodes and or energy markets. At most 8 nodes are allowed in a single request.
[14]:
def lmp_pull(directory,start_year,end_year,markets):
nodemap = dict(zip(directory.id, directory.name)) #takes the nodes dataframe and turns it into a dictionary
market_options = {'RT':Market.RT,'DA':Market.DA} #takes the list of markets to include so we can pull from the relevant database
results = {}
for market in markets:
print(f"Fetching LMP prices for market={market_options[market]}")
prices_response = lmp.get_prices(
node_ids=list(nodemap.keys()), #this is now a list of node ids (see node_directory)
market=market_options[market], #this pulls the right market
start_year=int(start_year),
end_year=int(end_year),
)
if prices_response.status_code == 200:
prices = prices_response.json()
results[market] = reformat_df(prices).rename(nodemap,axis=1) #this renames the column with the nodename instead of ID
else:
print(f"Error while pulling LMP prices: {prices_response._content}")
return pd.concat(results,axis=1)
Single Node Pull - RT & DA
Let’s pull the LMP data for the first node returned by our node search.
[15]:
single_node = node_directory.loc[node_directory.name== df.iloc[0]['node/name']]
single_node
[15]:
| da_start_year | rt_end_year | zone | name | type | substation | da_end_year | id | rt_start_year | |
|---|---|---|---|---|---|---|---|---|---|
| 613 | 2011.0 | 2025 | N/A | HB_BUSAVG | HUB | 2025.0 | 10000698380 | 2011 |
[16]:
lmps = lmp_pull(single_node,2019,2023,['DA','RT'])
lmps
Fetching LMP prices for market=Market.DA
Fetching LMP prices for market=Market.RT
[16]:
| DA | RT | |
|---|---|---|
| HB_BUSAVG | HB_BUSAVG | |
| 2019-01-01 00:00:00 | 20.49 | 13.7800 |
| 2019-01-01 01:00:00 | 19.77 | 14.8450 |
| 2019-01-01 02:00:00 | 19.56 | 15.4975 |
| 2019-01-01 03:00:00 | 19.21 | 16.1050 |
| 2019-01-01 04:00:00 | 21.22 | 16.0325 |
| ... | ... | ... |
| 2023-12-31 19:00:00 | 19.88 | 14.3475 |
| 2023-12-31 20:00:00 | 17.01 | 12.5775 |
| 2023-12-31 21:00:00 | 14.83 | 10.6650 |
| 2023-12-31 22:00:00 | 15.79 | 10.2950 |
| 2023-12-31 23:00:00 | 15.86 | 12.9600 |
43800 rows × 2 columns
Multi Node Pull - RT & DA
Let’s pull the LMP data for multiple nodes returned by our node search.
[17]:
df['node/name']
[17]:
0 HB_BUSAVG
1 HB_HOUSTON
2 HB_HUBAVG
3 HB_NORTH
4 HB_PAN
5 HB_SOUTH
6 HB_WEST
7 IHB_7_IRONHORSE_BATGNODE
8 PGHB_6_PDRP01-APND
9 PGHB_6_PDRP02-APND
10 PGHB_6_PDRP03-APND
11 PGHB_6_PDRP04-APND
12 PGHB_6_PDRP05-APND
13 PGHB_6_PDRP06-APND
14 PGHB_6_PDRP07-APND
15 PGHB_6_PDRP08-APND
16 PGHB_6_PDRP09-APND
17 PGHB_6_PDRP10-APND
18 PGHB_6_PDRP11-APND
19 PGHB_6_PDRP12-APND
20 PGHB_6_PDRP14-APND
21 PGHB_6_PDRP15-APND
22 PGHB_6_PDRP16-APND
23 PGHB_6_PDRP17-APND
24 PGHB_6_PDRP18-APND
Name: node/name, dtype: object
You can pull prices for at most 8 nodes in a single request. To pull prices for more than 8 nodes, split up the nodes into multiple requests.
[18]:
nodes_to_pull = df['node/name']
multiple_nodes = node_directory.loc[node_directory.name.isin(nodes_to_pull)]
multiple_nodes
[18]:
| da_start_year | rt_end_year | zone | name | type | substation | da_end_year | id | rt_start_year | |
|---|---|---|---|---|---|---|---|---|---|
| 30 | 2011.0 | 2025 | N/A | HB_HUBAVG | HUB | 2025.0 | 10000698382 | 2011 | |
| 95 | 2011.0 | 2025 | WEST | HB_WEST | HUB | 2025.0 | 10000697080 | 2011 | |
| 177 | 2011.0 | 2025 | NORTH | HB_NORTH | HUB | 2025.0 | 10000697078 | 2011 | |
| 285 | 2011.0 | 2025 | SOUTH | HB_SOUTH | HUB | 2025.0 | 10000697079 | 2011 | |
| 613 | 2011.0 | 2025 | N/A | HB_BUSAVG | HUB | 2025.0 | 10000698380 | 2011 | |
| 724 | 2011.0 | 2025 | HOUSTON | HB_HOUSTON | HUB | 2025.0 | 10000697077 | 2011 | |
| 857 | 2020.0 | 2025 | WEST | HB_PAN | HUB | 2025.0 | 10015999590 | 2020 |
[19]:
multiple_nodes_filtered = multiple_nodes[multiple_nodes['rt_start_year']<2020]
multiple_nodes_filtered = multiple_nodes_filtered[:8] # Limiting to 8 nodes
multiple_nodes_filtered
[19]:
| da_start_year | rt_end_year | zone | name | type | substation | da_end_year | id | rt_start_year | |
|---|---|---|---|---|---|---|---|---|---|
| 30 | 2011.0 | 2025 | N/A | HB_HUBAVG | HUB | 2025.0 | 10000698382 | 2011 | |
| 95 | 2011.0 | 2025 | WEST | HB_WEST | HUB | 2025.0 | 10000697080 | 2011 | |
| 177 | 2011.0 | 2025 | NORTH | HB_NORTH | HUB | 2025.0 | 10000697078 | 2011 | |
| 285 | 2011.0 | 2025 | SOUTH | HB_SOUTH | HUB | 2025.0 | 10000697079 | 2011 | |
| 613 | 2011.0 | 2025 | N/A | HB_BUSAVG | HUB | 2025.0 | 10000698380 | 2011 | |
| 724 | 2011.0 | 2025 | HOUSTON | HB_HOUSTON | HUB | 2025.0 | 10000697077 | 2011 |
[20]:
lmps_from_multiple = lmp_pull(multiple_nodes_filtered,2020,2024,['DA','RT'])
lmps_from_multiple
Fetching LMP prices for market=Market.DA
Fetching LMP prices for market=Market.RT
[20]:
| DA | RT | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HB_HUBAVG | HB_WEST | HB_NORTH | HB_SOUTH | HB_BUSAVG | HB_HOUSTON | HB_HUBAVG | HB_WEST | HB_NORTH | HB_SOUTH | HB_BUSAVG | HB_HOUSTON | |
| 2020-01-01 00:00:00 | 11.23 | 11.03 | 11.29 | 11.34 | 11.27 | 11.27 | 13.7075 | 13.7075 | 13.7075 | 13.7075 | 13.7075 | 13.7075 |
| 2020-01-01 01:00:00 | 9.99 | 9.83 | 10.01 | 10.40 | 10.04 | 9.73 | 13.2725 | 13.2725 | 13.2725 | 13.2725 | 13.2725 | 13.2725 |
| 2020-01-01 02:00:00 | 9.52 | 9.28 | 9.64 | 9.58 | 9.58 | 9.58 | 11.3700 | 11.3700 | 11.3700 | 11.3700 | 11.3700 | 11.3700 |
| 2020-01-01 03:00:00 | 9.89 | 9.67 | 9.95 | 10.30 | 9.95 | 9.65 | 9.4300 | 9.4250 | 9.4300 | 9.4375 | 9.4300 | 9.4325 |
| 2020-01-01 04:00:00 | 10.13 | 9.71 | 10.30 | 10.31 | 10.22 | 10.18 | 8.6325 | 8.6325 | 8.6325 | 8.6325 | 8.6325 | 8.6325 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 2024-12-31 19:00:00 | 34.31 | 40.37 | 35.70 | 28.68 | 34.25 | 32.47 | 34.7075 | 46.9075 | 36.9375 | 24.5675 | 34.5850 | 30.4125 |
| 2024-12-31 20:00:00 | 31.16 | 34.27 | 31.97 | 28.05 | 31.14 | 30.35 | 26.5600 | 28.7125 | 26.8275 | 24.8950 | 26.4975 | 25.8050 |
| 2024-12-31 21:00:00 | 25.26 | 27.07 | 25.64 | 23.51 | 25.22 | 24.81 | 22.7175 | 23.4000 | 22.5675 | 22.4250 | 22.6250 | 22.4825 |
| 2024-12-31 22:00:00 | 20.98 | 22.06 | 21.09 | 20.09 | 20.93 | 20.70 | 22.2400 | 22.2775 | 22.2300 | 22.2225 | 22.2325 | 22.2250 |
| 2024-12-31 23:00:00 | 20.39 | 21.12 | 20.39 | 19.97 | 20.34 | 20.11 | 20.2875 | 20.2875 | 20.2875 | 20.2875 | 20.2875 | 20.2875 |
43800 rows × 12 columns