{ "cells": [ { "cell_type": "markdown", "id": "bd2612f1ecfe01b0", "metadata": {}, "source": [ "# Node Search Price Pull Example" ] }, { "cell_type": "markdown", "id": "10f17acc-5746-485d-8d1b-2cb8ff13a4f3", "metadata": {}, "source": [ "## Notebook Overview" ] }, { "cell_type": "markdown", "id": "8c6f6b95-f896-416b-a696-3c4deaf16297", "metadata": {}, "source": [ "This notebook illustrates how to use the \"node search\" API endpoint. This endpoint allows users to search by location, node name, iso, or a combination. When searching by location, a list of the nearest nodes is returned. Once the user has identified the node they want to use, they can then pull historical prices for this node using the \"get prices\" API endpoint." ] }, { "cell_type": "markdown", "id": "fbca5408-ed14-4e4f-9223-963fbbd4a273", "metadata": { "tags": [] }, "source": [ "## Imports" ] }, { "cell_type": "markdown", "id": "e135df63-ebde-459a-ba32-dbcb00bfc199", "metadata": {}, "source": [ "First, let's import software packages, the Client API and your PAT." ] }, { "cell_type": "code", "execution_count": 1, "id": "813d463b-a160-4754-a8e4-3877c13545b0", "metadata": { "tags": [] }, "outputs": [], "source": [ "import time\n", "import os\n", "import pandas as pd\n", "from tyba_client.client import Client, Market\n", "\n", "PAT = os.environ[\"TYBA_PAT\"]\n", "client = Client(PAT)\n", "\n", "# Relevant API services\n", "services = client.services\n", "lmp = services.lmp\n", "ancillary = services.ancillary" ] }, { "cell_type": "markdown", "id": "b3f4fff3-dbe3-4899-b31e-8baf2da0c340", "metadata": {}, "source": [ "## Node Search Functions" ] }, { "cell_type": "markdown", "id": "c1b45998-4fef-48d1-acdc-b72db44bd9cc", "metadata": {}, "source": [ "Next, let's use the lmp.search_nodes function to search for a node. There are multiple ways a user can search for a node. To confirm successful search, status_code = 200.\n", "The function will return the node location, node ISO, node name, node ID, and node distance to the search location (if specified)." ] }, { "cell_type": "markdown", "id": "7b8e07a7-aaa2-4c85-8d63-62cb741d3f63", "metadata": {}, "source": [ "### 1. Search by location / address" ] }, { "cell_type": "markdown", "id": "82112399-c845-456e-86f1-19319c0676e4", "metadata": {}, "source": [ "Let's search by location and extract the list of nodes into a dataframe." ] }, { "cell_type": "code", "execution_count": 2, "id": "19933b1b-1940-4786-8454-40c84c610220", "metadata": { "tags": [] }, "outputs": [], "source": [ "res = lmp.search_nodes(location=\"dallas, texas\")\n", "res.raise_for_status()\n", "df = pd.DataFrame(res.json()[\"nodes\"])" ] }, { "cell_type": "code", "execution_count": 3, "id": "4b3c6115-a2a6-4049-b9e5-a10dad2a7281", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
node/isonode/namenode/idnode/distance-meters
0ERCOTEMPCT_RN100182670188198.673431
1ERCOTHB_NORTH1000069707812354.211404
2ERCOTLZ_NORTH1000069839212354.211404
3ERCOTPRCRK_RN1001826701612448.495097
4ERCOTMCSES_UNIT61000069867914209.794577
5ERCOTMCSES_UNIT71000069868014209.794577
6ERCOTMCSES_UNIT81000069868114209.794577
7ERCOTLIGSW_RN1001824846221842.460551
8ERCOTLHSES_UNIT11000069865924281.578207
9ERCOTLHSES_UNIT21000069866024281.578207
10ERCOTFRNYPP_1_CCU1000069858128839.581651
11ERCOTFRNYPP_2_CCU1000069858228839.581651
12ERCOTFRNYPP_CC11000069834328839.581651
13ERCOTFRNYPP_CC21000069834428839.581651
14ERCOTCNTRY_RN1001741931228907.071045
15ERCOTANOL_ESS_RN1001828749634680.822268
16ERCOTHLSES_UNIT31000069860839688.083699
17ERCOTHLSES_UNIT41000069860939688.083699
18ERCOTHLSES_UNIT51000069861039688.083699
19ERCOTELS_RN_11001696233342838.585540
20ERCOTLILY_RN1001649829143822.820037
21ERCOTDSWL_BES11001828749744573.640963
22ERCOTNOBLESLR_ALL1001687401644906.180346
23ERCOTMDANP_CT1_21000069868245619.085651
24ERCOTMDANP_CT3_41000069868345619.085651
\n", "
" ], "text/plain": [ " node/iso node/name node/id node/distance-meters\n", "0 ERCOT EMPCT_RN 10018267018 8198.673431\n", "1 ERCOT HB_NORTH 10000697078 12354.211404\n", "2 ERCOT LZ_NORTH 10000698392 12354.211404\n", "3 ERCOT PRCRK_RN 10018267016 12448.495097\n", "4 ERCOT MCSES_UNIT6 10000698679 14209.794577\n", "5 ERCOT MCSES_UNIT7 10000698680 14209.794577\n", "6 ERCOT MCSES_UNIT8 10000698681 14209.794577\n", "7 ERCOT LIGSW_RN 10018248462 21842.460551\n", "8 ERCOT LHSES_UNIT1 10000698659 24281.578207\n", "9 ERCOT LHSES_UNIT2 10000698660 24281.578207\n", "10 ERCOT FRNYPP_1_CCU 10000698581 28839.581651\n", "11 ERCOT FRNYPP_2_CCU 10000698582 28839.581651\n", "12 ERCOT FRNYPP_CC1 10000698343 28839.581651\n", "13 ERCOT FRNYPP_CC2 10000698344 28839.581651\n", "14 ERCOT CNTRY_RN 10017419312 28907.071045\n", "15 ERCOT ANOL_ESS_RN 10018287496 34680.822268\n", "16 ERCOT HLSES_UNIT3 10000698608 39688.083699\n", "17 ERCOT HLSES_UNIT4 10000698609 39688.083699\n", "18 ERCOT HLSES_UNIT5 10000698610 39688.083699\n", "19 ERCOT ELS_RN_1 10016962333 42838.585540\n", "20 ERCOT LILY_RN 10016498291 43822.820037\n", "21 ERCOT DSWL_BES1 10018287497 44573.640963\n", "22 ERCOT NOBLESLR_ALL 10016874016 44906.180346\n", "23 ERCOT MDANP_CT1_2 10000698682 45619.085651\n", "24 ERCOT MDANP_CT3_4 10000698683 45619.085651" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df" ] }, { "cell_type": "markdown", "id": "fba48c29-8dda-47b1-8238-495c9340c9cb", "metadata": {}, "source": [ "We can also search by lat/lng or street address." ] }, { "cell_type": "code", "execution_count": 4, "id": "55efa22a-955c-42c1-9be1-8af3921d1bc5", "metadata": { "tags": [] }, "outputs": [], "source": [ "res = lmp.search_nodes(location=\"29.760427, -95.369804\")\n", "res = lmp.search_nodes(location=\"3 NRG Pkwy, Houston, TX 77054\")" ] }, { "cell_type": "markdown", "id": "a849dcb9-bc41-43ef-8267-7f756a22c428", "metadata": {}, "source": [ "### 2. Search by portion of node name" ] }, { "cell_type": "code", "execution_count": 5, "id": "049e9fd3-a82f-447b-8335-c35e6e9ca913", "metadata": { "tags": [] }, "outputs": [], "source": [ "res = lmp.search_nodes(node_name_filter=\"HB_\")\n", "res.raise_for_status()\n", "df = pd.DataFrame(res.json()[\"nodes\"])" ] }, { "cell_type": "code", "execution_count": 6, "id": "390553bd-2d1c-499d-9790-67932c9268a0", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
node/idnode/namenode/iso
010000698380HB_BUSAVGERCOT
110000697077HB_HOUSTONERCOT
210000698382HB_HUBAVGERCOT
310000697078HB_NORTHERCOT
410015999590HB_PANERCOT
510000697079HB_SOUTHERCOT
610000697080HB_WESTERCOT
710016973502IHB_7_IRONHORSE_BATGNODECAISO
810004176539PGHB_6_PDRP01-APNDCAISO
910004074430PGHB_6_PDRP02-APNDCAISO
1010002701851PGHB_6_PDRP03-APNDCAISO
1110004262229PGHB_6_PDRP04-APNDCAISO
1210015847591PGHB_6_PDRP05-APNDCAISO
1310015853877PGHB_6_PDRP06-APNDCAISO
1410015880548PGHB_6_PDRP07-APNDCAISO
1510016481077PGHB_6_PDRP08-APNDCAISO
1610016481089PGHB_6_PDRP09-APNDCAISO
1710016019849PGHB_6_PDRP10-APNDCAISO
1810016240024PGHB_6_PDRP11-APNDCAISO
1910016240072PGHB_6_PDRP12-APNDCAISO
2010016481091PGHB_6_PDRP14-APNDCAISO
2110016481092PGHB_6_PDRP15-APNDCAISO
2210016481078PGHB_6_PDRP16-APNDCAISO
2310016495007PGHB_6_PDRP17-APNDCAISO
2410016656810PGHB_6_PDRP18-APNDCAISO
\n", "
" ], "text/plain": [ " node/id node/name node/iso\n", "0 10000698380 HB_BUSAVG ERCOT\n", "1 10000697077 HB_HOUSTON ERCOT\n", "2 10000698382 HB_HUBAVG ERCOT\n", "3 10000697078 HB_NORTH ERCOT\n", "4 10015999590 HB_PAN ERCOT\n", "5 10000697079 HB_SOUTH ERCOT\n", "6 10000697080 HB_WEST ERCOT\n", "7 10016973502 IHB_7_IRONHORSE_BATGNODE CAISO\n", "8 10004176539 PGHB_6_PDRP01-APND CAISO\n", "9 10004074430 PGHB_6_PDRP02-APND CAISO\n", "10 10002701851 PGHB_6_PDRP03-APND CAISO\n", "11 10004262229 PGHB_6_PDRP04-APND CAISO\n", "12 10015847591 PGHB_6_PDRP05-APND CAISO\n", "13 10015853877 PGHB_6_PDRP06-APND CAISO\n", "14 10015880548 PGHB_6_PDRP07-APND CAISO\n", "15 10016481077 PGHB_6_PDRP08-APND CAISO\n", "16 10016481089 PGHB_6_PDRP09-APND CAISO\n", "17 10016019849 PGHB_6_PDRP10-APND CAISO\n", "18 10016240024 PGHB_6_PDRP11-APND CAISO\n", "19 10016240072 PGHB_6_PDRP12-APND CAISO\n", "20 10016481091 PGHB_6_PDRP14-APND CAISO\n", "21 10016481092 PGHB_6_PDRP15-APND CAISO\n", "22 10016481078 PGHB_6_PDRP16-APND CAISO\n", "23 10016495007 PGHB_6_PDRP17-APND CAISO\n", "24 10016656810 PGHB_6_PDRP18-APND CAISO" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df" ] }, { "cell_type": "markdown", "id": "1cfdc52c-5b54-4049-9dc0-3fa3b7143d4e", "metadata": {}, "source": [ "### 3. Search by portion of node name and location / address" ] }, { "cell_type": "markdown", "id": "197214e1-1be8-42fd-ad19-4c448f9c3889", "metadata": {}, "source": [ "We can combine multiple search filters to narrow down further." ] }, { "cell_type": "code", "execution_count": 7, "id": "47796284-2419-4426-8f9c-2156b4e159fb", "metadata": { "tags": [] }, "outputs": [], "source": [ "res = lmp.search_nodes(node_name_filter=\"HB_\", location=\"Houston, Texas\")" ] }, { "cell_type": "markdown", "id": "33103c4d-51fb-4262-8dcc-2d1374e4c973", "metadata": {}, "source": [ "### 4. Search for a location but with an ISO override to constrain search to the specified ISO" ] }, { "cell_type": "markdown", "id": "44fbca04-6bcd-437c-8565-8270b7bcc981", "metadata": {}, "source": [ "We can even search for a location but then specify an ISO within which to look for nodes." ] }, { "cell_type": "code", "execution_count": 8, "id": "c290abb2-4379-4321-b147-2b668e0d2080", "metadata": { "tags": [] }, "outputs": [], "source": [ "res = lmp.search_nodes(location=\"toledo, OH\", iso_override=\"MISO\")" ] }, { "cell_type": "markdown", "id": "e0b33948-cde3-45a3-b474-4b88b9109c3f", "metadata": {}, "source": [ "## Pull Energy Prices" ] }, { "cell_type": "markdown", "id": "222efa9a-2f14-47e0-a335-1b0cba40ccee", "metadata": {}, "source": [ "Now that we've searched for the node(s) in question, let's extract the relevant node directory information." ] }, { "cell_type": "code", "execution_count": 9, "id": "e3177b36-f65f-4e39-831e-8491db348a8c", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
node/idnode/namenode/iso
010000698380HB_BUSAVGERCOT
110000697077HB_HOUSTONERCOT
210000698382HB_HUBAVGERCOT
310000697078HB_NORTHERCOT
410015999590HB_PANERCOT
510000697079HB_SOUTHERCOT
610000697080HB_WESTERCOT
710016973502IHB_7_IRONHORSE_BATGNODECAISO
810004176539PGHB_6_PDRP01-APNDCAISO
910004074430PGHB_6_PDRP02-APNDCAISO
1010002701851PGHB_6_PDRP03-APNDCAISO
1110004262229PGHB_6_PDRP04-APNDCAISO
1210015847591PGHB_6_PDRP05-APNDCAISO
1310015853877PGHB_6_PDRP06-APNDCAISO
1410015880548PGHB_6_PDRP07-APNDCAISO
1510016481077PGHB_6_PDRP08-APNDCAISO
1610016481089PGHB_6_PDRP09-APNDCAISO
1710016019849PGHB_6_PDRP10-APNDCAISO
1810016240024PGHB_6_PDRP11-APNDCAISO
1910016240072PGHB_6_PDRP12-APNDCAISO
2010016481091PGHB_6_PDRP14-APNDCAISO
2110016481092PGHB_6_PDRP15-APNDCAISO
2210016481078PGHB_6_PDRP16-APNDCAISO
2310016495007PGHB_6_PDRP17-APNDCAISO
2410016656810PGHB_6_PDRP18-APNDCAISO
\n", "
" ], "text/plain": [ " node/id node/name node/iso\n", "0 10000698380 HB_BUSAVG ERCOT\n", "1 10000697077 HB_HOUSTON ERCOT\n", "2 10000698382 HB_HUBAVG ERCOT\n", "3 10000697078 HB_NORTH ERCOT\n", "4 10015999590 HB_PAN ERCOT\n", "5 10000697079 HB_SOUTH ERCOT\n", "6 10000697080 HB_WEST ERCOT\n", "7 10016973502 IHB_7_IRONHORSE_BATGNODE CAISO\n", "8 10004176539 PGHB_6_PDRP01-APND CAISO\n", "9 10004074430 PGHB_6_PDRP02-APND CAISO\n", "10 10002701851 PGHB_6_PDRP03-APND CAISO\n", "11 10004262229 PGHB_6_PDRP04-APND CAISO\n", "12 10015847591 PGHB_6_PDRP05-APND CAISO\n", "13 10015853877 PGHB_6_PDRP06-APND CAISO\n", "14 10015880548 PGHB_6_PDRP07-APND CAISO\n", "15 10016481077 PGHB_6_PDRP08-APND CAISO\n", "16 10016481089 PGHB_6_PDRP09-APND CAISO\n", "17 10016019849 PGHB_6_PDRP10-APND CAISO\n", "18 10016240024 PGHB_6_PDRP11-APND CAISO\n", "19 10016240072 PGHB_6_PDRP12-APND CAISO\n", "20 10016481091 PGHB_6_PDRP14-APND CAISO\n", "21 10016481092 PGHB_6_PDRP15-APND CAISO\n", "22 10016481078 PGHB_6_PDRP16-APND CAISO\n", "23 10016495007 PGHB_6_PDRP17-APND CAISO\n", "24 10016656810 PGHB_6_PDRP18-APND CAISO" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df" ] }, { "cell_type": "markdown", "id": "d9af61e1-74ba-4d24-8fb8-86b1c12390ec", "metadata": {}, "source": [ "### Pull all ISO nodes" ] }, { "cell_type": "markdown", "id": "d3509ca0-3afa-418d-bc9c-774b3dcac6b8", "metadata": {}, "source": [ "Let's first pull the ERCOT ISO directory. We'll filter this directory to use in any energy price pulls." ] }, { "cell_type": "code", "execution_count": 10, "id": "ea3c7a0d-f6ae-4303-888b-6f79fb698fc5", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "['PJMISO', 'CAISO', 'NEISO', 'NYISO', 'ERCOT', 'SPPISO', 'MISO']" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "services.get_all_isos().json()" ] }, { "cell_type": "code", "execution_count": 11, "id": "d89a43ee-3fba-4ea4-9fde-944450ec5edb", "metadata": { "tags": [] }, "outputs": [], "source": [ "price_nodes = lmp.get_all_nodes(iso='ERCOT').json()\n", "node_directory = pd.DataFrame(price_nodes)" ] }, { "cell_type": "code", "execution_count": 12, "id": "a7ea7f22", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
da_start_yearrt_end_yearzonenametypesubstationda_end_yearidrt_start_year
02011.02025SOUTHOXY_OXY_CCG1GENERATOROXYCHEM CC2025.0100006987212011
12022.02025SOUTHSLCN_ESS_ALLGENERATORNaN2025.0100166717262022
22024.02025SOUTHANEM_ESS_RNGENERATORNaN2025.0100173188332024
32011.02025NORTHWHCCS_CC1GENERATORWOLF HOLLOW GEN2025.0100006983792011
42011.02025HOUSTONSCLP_PUN4GENERATORSWEENY COGEN2025.0100006987802011
..............................
10712022.02025WESTQUEEN_BA_RNGENERATORNaN2025.0100168740152022
10722024.02025HOUSTONRBN_BESS1LOADNaN2025.0100172900642024
10732018.02025SOUTHSANTACRU_ALLGENERATORSANTACRU2025.0100041838832018
10742023.02025WESTPALE_ESS_ENGENERATORNaN2025.0100171773792023
10752011.02025HOUSTONWAP_WAP_G2GENERATORWA PARISH2025.0100006988702011
\n", "

1076 rows × 9 columns

\n", "
" ], "text/plain": [ " da_start_year rt_end_year zone name type \\\n", "0 2011.0 2025 SOUTH OXY_OXY_CCG1 GENERATOR \n", "1 2022.0 2025 SOUTH SLCN_ESS_ALL GENERATOR \n", "2 2024.0 2025 SOUTH ANEM_ESS_RN GENERATOR \n", "3 2011.0 2025 NORTH WHCCS_CC1 GENERATOR \n", "4 2011.0 2025 HOUSTON SCLP_PUN4 GENERATOR \n", "... ... ... ... ... ... \n", "1071 2022.0 2025 WEST QUEEN_BA_RN GENERATOR \n", "1072 2024.0 2025 HOUSTON RBN_BESS1 LOAD \n", "1073 2018.0 2025 SOUTH SANTACRU_ALL GENERATOR \n", "1074 2023.0 2025 WEST PALE_ESS_EN GENERATOR \n", "1075 2011.0 2025 HOUSTON WAP_WAP_G2 GENERATOR \n", "\n", " substation da_end_year id rt_start_year \n", "0 OXYCHEM CC 2025.0 10000698721 2011 \n", "1 NaN 2025.0 10016671726 2022 \n", "2 NaN 2025.0 10017318833 2024 \n", "3 WOLF HOLLOW GEN 2025.0 10000698379 2011 \n", "4 SWEENY COGEN 2025.0 10000698780 2011 \n", "... ... ... ... ... \n", "1071 NaN 2025.0 10016874015 2022 \n", "1072 NaN 2025.0 10017290064 2024 \n", "1073 SANTACRU 2025.0 10004183883 2018 \n", "1074 NaN 2025.0 10017177379 2023 \n", "1075 WA PARISH 2025.0 10000698870 2011 \n", "\n", "[1076 rows x 9 columns]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "node_directory" ] }, { "cell_type": "markdown", "id": "e2deaa82-c2c2-409a-9c2a-fafafe6a18bf", "metadata": { "tags": [] }, "source": [ "### Helper Functions" ] }, { "cell_type": "markdown", "id": "5315d731-90d9-4357-bce9-ea94a7a81334", "metadata": {}, "source": [ "This takes the raw API payload and converts it into a dataframe with a datetime index:" ] }, { "cell_type": "code", "execution_count": 13, "id": "e1db68a9-0574-4ad1-a496-54d92d62aad8", "metadata": { "tags": [] }, "outputs": [], "source": [ "def reformat_df(result):\n", " df = pd.concat({node:pd.DataFrame(results['prices'],index=pd.to_datetime(results['datetimes'])) for node,results in result.items()},axis=1)\n", " return df.droplevel(axis=1,level=1).tz_localize(None)" ] }, { "cell_type": "markdown", "id": "8b1511f9-8e87-4d44-96d3-674dfae70be9", "metadata": {}, "source": [ "This is a function to simplify pulling 1 or more nodes and or energy markets. At most 8 nodes are allowed in a single request." ] }, { "cell_type": "code", "execution_count": 14, "id": "01f7848e-582e-4e11-868c-6a2c2cdb1875", "metadata": { "tags": [] }, "outputs": [], "source": [ "def lmp_pull(directory,start_year,end_year,markets): \n", " \n", " nodemap = dict(zip(directory.id, directory.name)) #takes the nodes dataframe and turns it into a dictionary\n", " market_options = {'RT':Market.RT,'DA':Market.DA} #takes the list of markets to include so we can pull from the relevant database\n", " \n", " results = {}\n", " for market in markets:\n", " print(f\"Fetching LMP prices for market={market_options[market]}\")\n", " prices_response = lmp.get_prices(\n", " node_ids=list(nodemap.keys()), #this is now a list of node ids (see node_directory)\n", " market=market_options[market], #this pulls the right market\n", " start_year=int(start_year), \n", " end_year=int(end_year),\n", " )\n", "\n", " if prices_response.status_code == 200:\n", " prices = prices_response.json()\n", " results[market] = reformat_df(prices).rename(nodemap,axis=1) #this renames the column with the nodename instead of ID\n", " else:\n", " print(f\"Error while pulling LMP prices: {prices_response._content}\")\n", "\n", " return pd.concat(results,axis=1)\n", " " ] }, { "cell_type": "markdown", "id": "41766d39-a9cd-4306-89e5-82d988a3970e", "metadata": {}, "source": [ "### Single Node Pull - RT & DA" ] }, { "cell_type": "markdown", "id": "4646a67b-8194-443a-95f2-afa14156975a", "metadata": {}, "source": [ "Let's pull the LMP data for the first node returned by our node search." ] }, { "cell_type": "code", "execution_count": 15, "id": "3e21a324-502e-4510-9e39-173a37f5b1d5", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
da_start_yearrt_end_yearzonenametypesubstationda_end_yearidrt_start_year
6132011.02025N/AHB_BUSAVGHUB2025.0100006983802011
\n", "
" ], "text/plain": [ " da_start_year rt_end_year zone name type substation da_end_year \\\n", "613 2011.0 2025 N/A HB_BUSAVG HUB 2025.0 \n", "\n", " id rt_start_year \n", "613 10000698380 2011 " ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "single_node = node_directory.loc[node_directory.name== df.iloc[0]['node/name']]\n", "single_node" ] }, { "cell_type": "code", "execution_count": 16, "id": "f96a8b8a-83c4-41d6-940c-433de3bdecba", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Fetching LMP prices for market=Market.DA\n", "Fetching LMP prices for market=Market.RT\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
DART
HB_BUSAVGHB_BUSAVG
2019-01-01 00:00:0020.4913.7800
2019-01-01 01:00:0019.7714.8450
2019-01-01 02:00:0019.5615.4975
2019-01-01 03:00:0019.2116.1050
2019-01-01 04:00:0021.2216.0325
.........
2023-12-31 19:00:0019.8814.3475
2023-12-31 20:00:0017.0112.5775
2023-12-31 21:00:0014.8310.6650
2023-12-31 22:00:0015.7910.2950
2023-12-31 23:00:0015.8612.9600
\n", "

43800 rows × 2 columns

\n", "
" ], "text/plain": [ " DA RT\n", " HB_BUSAVG HB_BUSAVG\n", "2019-01-01 00:00:00 20.49 13.7800\n", "2019-01-01 01:00:00 19.77 14.8450\n", "2019-01-01 02:00:00 19.56 15.4975\n", "2019-01-01 03:00:00 19.21 16.1050\n", "2019-01-01 04:00:00 21.22 16.0325\n", "... ... ...\n", "2023-12-31 19:00:00 19.88 14.3475\n", "2023-12-31 20:00:00 17.01 12.5775\n", "2023-12-31 21:00:00 14.83 10.6650\n", "2023-12-31 22:00:00 15.79 10.2950\n", "2023-12-31 23:00:00 15.86 12.9600\n", "\n", "[43800 rows x 2 columns]" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lmps = lmp_pull(single_node,2019,2023,['DA','RT'])\n", "lmps" ] }, { "cell_type": "markdown", "id": "fd2ec0b4-c3c3-48c9-a687-1424d1fe887c", "metadata": { "tags": [] }, "source": [ "### Multi Node Pull - RT & DA" ] }, { "cell_type": "markdown", "id": "5e115d15-9748-4e13-8a29-f241c1a9570b", "metadata": {}, "source": [ "Let's pull the LMP data for multiple nodes returned by our node search." ] }, { "cell_type": "code", "execution_count": 17, "id": "a14c8193-a6c6-4fd7-9151-1db39b6f352c", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/plain": [ "0 HB_BUSAVG\n", "1 HB_HOUSTON\n", "2 HB_HUBAVG\n", "3 HB_NORTH\n", "4 HB_PAN\n", "5 HB_SOUTH\n", "6 HB_WEST\n", "7 IHB_7_IRONHORSE_BATGNODE\n", "8 PGHB_6_PDRP01-APND\n", "9 PGHB_6_PDRP02-APND\n", "10 PGHB_6_PDRP03-APND\n", "11 PGHB_6_PDRP04-APND\n", "12 PGHB_6_PDRP05-APND\n", "13 PGHB_6_PDRP06-APND\n", "14 PGHB_6_PDRP07-APND\n", "15 PGHB_6_PDRP08-APND\n", "16 PGHB_6_PDRP09-APND\n", "17 PGHB_6_PDRP10-APND\n", "18 PGHB_6_PDRP11-APND\n", "19 PGHB_6_PDRP12-APND\n", "20 PGHB_6_PDRP14-APND\n", "21 PGHB_6_PDRP15-APND\n", "22 PGHB_6_PDRP16-APND\n", "23 PGHB_6_PDRP17-APND\n", "24 PGHB_6_PDRP18-APND\n", "Name: node/name, dtype: object" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df['node/name']" ] }, { "cell_type": "markdown", "id": "f9fa3da0-0a4d-4165-8f54-3b989082231b", "metadata": {}, "source": [ "You can pull prices for at most 8 nodes in a single request. To pull prices for more than 8 nodes, split up the nodes into multiple requests." ] }, { "cell_type": "code", "execution_count": 18, "id": "bd918c69-23e6-4152-9bed-f15a1f5bab1c", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
da_start_yearrt_end_yearzonenametypesubstationda_end_yearidrt_start_year
302011.02025N/AHB_HUBAVGHUB2025.0100006983822011
952011.02025WESTHB_WESTHUB2025.0100006970802011
1772011.02025NORTHHB_NORTHHUB2025.0100006970782011
2852011.02025SOUTHHB_SOUTHHUB2025.0100006970792011
6132011.02025N/AHB_BUSAVGHUB2025.0100006983802011
7242011.02025HOUSTONHB_HOUSTONHUB2025.0100006970772011
8572020.02025WESTHB_PANHUB2025.0100159995902020
\n", "
" ], "text/plain": [ " da_start_year rt_end_year zone name type substation \\\n", "30 2011.0 2025 N/A HB_HUBAVG HUB \n", "95 2011.0 2025 WEST HB_WEST HUB \n", "177 2011.0 2025 NORTH HB_NORTH HUB \n", "285 2011.0 2025 SOUTH HB_SOUTH HUB \n", "613 2011.0 2025 N/A HB_BUSAVG HUB \n", "724 2011.0 2025 HOUSTON HB_HOUSTON HUB \n", "857 2020.0 2025 WEST HB_PAN HUB \n", "\n", " da_end_year id rt_start_year \n", "30 2025.0 10000698382 2011 \n", "95 2025.0 10000697080 2011 \n", "177 2025.0 10000697078 2011 \n", "285 2025.0 10000697079 2011 \n", "613 2025.0 10000698380 2011 \n", "724 2025.0 10000697077 2011 \n", "857 2025.0 10015999590 2020 " ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "nodes_to_pull = df['node/name']\n", "\n", "multiple_nodes = node_directory.loc[node_directory.name.isin(nodes_to_pull)]\n", "multiple_nodes" ] }, { "cell_type": "code", "execution_count": 19, "id": "62a87a4a-ffd6-41ad-9b5b-fe63b2f3fd9b", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
da_start_yearrt_end_yearzonenametypesubstationda_end_yearidrt_start_year
302011.02025N/AHB_HUBAVGHUB2025.0100006983822011
952011.02025WESTHB_WESTHUB2025.0100006970802011
1772011.02025NORTHHB_NORTHHUB2025.0100006970782011
2852011.02025SOUTHHB_SOUTHHUB2025.0100006970792011
6132011.02025N/AHB_BUSAVGHUB2025.0100006983802011
7242011.02025HOUSTONHB_HOUSTONHUB2025.0100006970772011
\n", "
" ], "text/plain": [ " da_start_year rt_end_year zone name type substation \\\n", "30 2011.0 2025 N/A HB_HUBAVG HUB \n", "95 2011.0 2025 WEST HB_WEST HUB \n", "177 2011.0 2025 NORTH HB_NORTH HUB \n", "285 2011.0 2025 SOUTH HB_SOUTH HUB \n", "613 2011.0 2025 N/A HB_BUSAVG HUB \n", "724 2011.0 2025 HOUSTON HB_HOUSTON HUB \n", "\n", " da_end_year id rt_start_year \n", "30 2025.0 10000698382 2011 \n", "95 2025.0 10000697080 2011 \n", "177 2025.0 10000697078 2011 \n", "285 2025.0 10000697079 2011 \n", "613 2025.0 10000698380 2011 \n", "724 2025.0 10000697077 2011 " ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "multiple_nodes_filtered = multiple_nodes[multiple_nodes['rt_start_year']<2020]\n", "\n", "multiple_nodes_filtered = multiple_nodes_filtered[:8] # Limiting to 8 nodes\n", "multiple_nodes_filtered" ] }, { "cell_type": "code", "execution_count": 20, "id": "fd795009-9812-4030-beb3-65faa3625240", "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Fetching LMP prices for market=Market.DA\n", "Fetching LMP prices for market=Market.RT\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
DART
HB_HUBAVGHB_WESTHB_NORTHHB_SOUTHHB_BUSAVGHB_HOUSTONHB_HUBAVGHB_WESTHB_NORTHHB_SOUTHHB_BUSAVGHB_HOUSTON
2020-01-01 00:00:0011.2311.0311.2911.3411.2711.2713.707513.707513.707513.707513.707513.7075
2020-01-01 01:00:009.999.8310.0110.4010.049.7313.272513.272513.272513.272513.272513.2725
2020-01-01 02:00:009.529.289.649.589.589.5811.370011.370011.370011.370011.370011.3700
2020-01-01 03:00:009.899.679.9510.309.959.659.43009.42509.43009.43759.43009.4325
2020-01-01 04:00:0010.139.7110.3010.3110.2210.188.63258.63258.63258.63258.63258.6325
.......................................
2024-12-31 19:00:0034.3140.3735.7028.6834.2532.4734.707546.907536.937524.567534.585030.4125
2024-12-31 20:00:0031.1634.2731.9728.0531.1430.3526.560028.712526.827524.895026.497525.8050
2024-12-31 21:00:0025.2627.0725.6423.5125.2224.8122.717523.400022.567522.425022.625022.4825
2024-12-31 22:00:0020.9822.0621.0920.0920.9320.7022.240022.277522.230022.222522.232522.2250
2024-12-31 23:00:0020.3921.1220.3919.9720.3420.1120.287520.287520.287520.287520.287520.2875
\n", "

43800 rows × 12 columns

\n", "
" ], "text/plain": [ " DA \\\n", " HB_HUBAVG HB_WEST HB_NORTH HB_SOUTH HB_BUSAVG HB_HOUSTON \n", "2020-01-01 00:00:00 11.23 11.03 11.29 11.34 11.27 11.27 \n", "2020-01-01 01:00:00 9.99 9.83 10.01 10.40 10.04 9.73 \n", "2020-01-01 02:00:00 9.52 9.28 9.64 9.58 9.58 9.58 \n", "2020-01-01 03:00:00 9.89 9.67 9.95 10.30 9.95 9.65 \n", "2020-01-01 04:00:00 10.13 9.71 10.30 10.31 10.22 10.18 \n", "... ... ... ... ... ... ... \n", "2024-12-31 19:00:00 34.31 40.37 35.70 28.68 34.25 32.47 \n", "2024-12-31 20:00:00 31.16 34.27 31.97 28.05 31.14 30.35 \n", "2024-12-31 21:00:00 25.26 27.07 25.64 23.51 25.22 24.81 \n", "2024-12-31 22:00:00 20.98 22.06 21.09 20.09 20.93 20.70 \n", "2024-12-31 23:00:00 20.39 21.12 20.39 19.97 20.34 20.11 \n", "\n", " RT \n", " HB_HUBAVG HB_WEST HB_NORTH HB_SOUTH HB_BUSAVG HB_HOUSTON \n", "2020-01-01 00:00:00 13.7075 13.7075 13.7075 13.7075 13.7075 13.7075 \n", "2020-01-01 01:00:00 13.2725 13.2725 13.2725 13.2725 13.2725 13.2725 \n", "2020-01-01 02:00:00 11.3700 11.3700 11.3700 11.3700 11.3700 11.3700 \n", "2020-01-01 03:00:00 9.4300 9.4250 9.4300 9.4375 9.4300 9.4325 \n", "2020-01-01 04:00:00 8.6325 8.6325 8.6325 8.6325 8.6325 8.6325 \n", "... ... ... ... ... ... ... \n", "2024-12-31 19:00:00 34.7075 46.9075 36.9375 24.5675 34.5850 30.4125 \n", "2024-12-31 20:00:00 26.5600 28.7125 26.8275 24.8950 26.4975 25.8050 \n", "2024-12-31 21:00:00 22.7175 23.4000 22.5675 22.4250 22.6250 22.4825 \n", "2024-12-31 22:00:00 22.2400 22.2775 22.2300 22.2225 22.2325 22.2250 \n", "2024-12-31 23:00:00 20.2875 20.2875 20.2875 20.2875 20.2875 20.2875 \n", "\n", "[43800 rows x 12 columns]" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lmps_from_multiple = lmp_pull(multiple_nodes_filtered,2020,2024,['DA','RT'])\n", "lmps_from_multiple" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.11" } }, "nbformat": 4, "nbformat_minor": 5 }