Chpater 5.4: PROMOD Benchmarking Study

Chpater 5.4: PROMOD Benchmarking Study

The PROMOD Benchmark Study analyzes differences between the MISO market and PROMOD simulation tool, and identifies best modeling practices to enhance the accuracy and capability of the simulation tool in both backward- and forward-looking analyses.

The study started in 2014, and therefore 2013 was chosen as the most current focus year. A historical-looking PROMOD model, termed Base PROMOD Model, was built with 2013 data for load, gas prices, generation fleet and the transmission system. The simulation results of the Base PROMOD Model showed drastic differences with actual MISO market outcome, especially in regards to transmission system congestion and locational marginal price (LMP).

Significant efforts were spent identifying the causes of the differences and improving the model to minimize these differences. After applying various modeling changes, the Final PROMOD Model was able to capture 76 percent of the 2013 MISO Day-ahead Market congestion in terms of total shadow price, a nearly three-fold improvement in congestion from the 28 percent congestion captured in the Base PROMOD Model.

This substantial improvement came as a result of a number of modeling changes identified and implemented in the PROMOD Benchmark Study. To help understand the impact of these changes and identify the major contributing factors, they are classified into these categories:

  • Generation Outages
  • Generation Characteristic Changes
  • Transmission Outages
  • Transmission Derates
  • Other Modeling Improvements
  • Renewable Energy Updates
  • Pool Interchange Lockdown

 

Each category impacts MISO transmission system congestion differently (Figure 5.4-1). For example, the transmission outages category has the biggest impact at 18 percent, which means modeling transmission outages captured an incremental 18 percent of the 2013 MISO Day-ahead Market congestion. Other categories with significant impact are generation characteristic changes, generation outages, other modeling improvements and pool interchange lockdown.

Figure 5.4-1: Congestion impact of modeling changes in the benchmark study

Figure 5.4-1: Congestion impact of modeling changes in the benchmark study

The remaining difference of 24 percent can be attributed to various potential reasons, some of which may require enhancement of the simulation software itself.

Out of all the changes implemented in the PROMOD Benchmark Study, about 40 percent of the improvements are the result of modeling changes that may be applied to forward-looking analyses. The applicable modeling updates will be vetted through the stakeholder process before incorporation into future planning studies. These updates involve generator modeling, transmission limit adjustment, non-conforming load modeling and specific phase angle regulator modeling.

Study Process

The PROMOD Benchmark study process consisted of gathering historical information from both public and proprietary sources, analyzing the historical congestion pattern, comparing historical actuals with simulated results from PROMOD including generation and line flow, identifying discrepancies and potential causes, and eventually developing modeling changes and performing PROMOD simulations to verify the impact of the changes. Due to the complex nature of the issue, the differences seen between PROMOD simulation results and historical actuals usually stem from a multitude of causes rather than a single cause. If the simulation does not show enough improvements or shows unexpected results, additional information is collected, typically on a more granular level, to investigate the issue further and develop refined modeling changes for further testing. Therefore, the process is highly iterative (Figure 5.4-2).

Figure 5.4-2: Benchmark study process

Figure 5.4-2: Benchmark study process

Summary of Modeling Changes

The differences between PROMOD simulation results and MISO market actuals come from three different sources: input data accuracy and granularity; modeling approach and implementation; and inherent difference between PROMOD and the market. These differences manifest in many ways including market structure, simulation footprint, commitment and dispatch, modeling of generation, load, transmission, fuel, interchange and external areas. Each has varying levels of impact. The PROMOD Benchmark Study analyzed these differences in significant detail to identify modeling changes and needed enhancements to the PROMOD tool.

The various modeling changes implemented in the PROMOD Benchmark Study are categorized into a few groups (Figure 5.4-1), and each group of changes is elaborated on as follows:

Generation Outages

The PROMOD Benchmark Study modeled actual MISO North/Central region generator outages, including both planned and forced outages for 2013. Some generator outages in external areas were also modeled. The impact of these changes depended on the location of the generator outage relative to the constraint. Overall, by modeling the generation outages, an additional 9 percent of 2013 MISO day-ahead congestion was captured.

Generation Characteristic Changes

Using data from various sources, the operating characteristics of many generation units were modified, such as the unit’s heat rate, minimum capacity and maximum capacity. This change also included modifying the must-run statuses of various coal-fired and combined-cycle units for MISO and some neighboring areas based on historical data. This category of changes put generation output more in line with actual 2013 historical generation output. Overall, modifying these characteristics captured an additional 16 percent of 2013 MISO day-ahead congestion.

Transmission Outages

The study modeled the majority of its 2013 transmission outages, including both planned and forced outages, as well as some PJM 2013 outages. It should be noted that all transmission outages were modeled as planned outages in PROMOD due to its capability. Because powerflow can change significantly as a result of transmission outages, modeling these outages can have a dramatic effect on the congestion of specific flowgates. Overall, modeling of transmission outages captured an additional 18 percent of 2013 MISO day-ahead congestion.

Transmission Derates

Based on historical information, the study updated limits for flowgates in some focused areas. This generally increased congestion as it involved various rating decreases. It captured an additional 4 percent of 2013 MISO day-ahead congestion. The impact is more dramatic in the focused areas. For instance, the modeling of transmission derates captured an additional 22 percent of 2013 Northern Indiana Public Service Co.’s (NIPSCO) day-ahead congestion and an additional 13 percent of 2013 Ameren Illinois (AMIL) day-ahead congestion.

Other Modeling Improvements

This category of changes includes various modeling updates that do not fall under the rest of the categories. This includes non-conforming load modeling in some MISO and PJM areas, specific phase angle regulator modeling improvement and coal price update. Among these changes, non-conforming load modeling updates improved the distribution of congestion on flowgates. The impact of this category of changes is an additional 9 percent of 2013 MISO day-ahead congestion captured.

Renewable Energy Update

This change set the total amount of wind energy of MISO and PJM to actual wind energy for 2013. In the Base PROMOD Model, MISO had more wind energy modeled than historical, setting MISO wind energy to the actual historical amount tended to reduce congestion due to less west-to-east flow. At the same time, PJM had less wind energy modeled in the Base PROMOD Model than the actual historical amount, and therefore setting PJM wind energy to historical amounts increased congestion, particularly, in NIPSCO area. As a result of these changes, NIPSCO congestion increased by 11 percent of its 2013 day-ahead level, and AMIL congestion decreased by 8 percent of its 2013 day-ahead level. Overall, an additional 3 percent of 2013 MISO day-ahead congestion is captured.

Pool Interchange Lockdown

This change set the interchange between MISO and all neighboring pools at the actual historical interchange of 2013. This increased congestion on flowgates at or near the seams that were relevant to meeting these interchange schedules. Overall, modeling of the pool interchange lockdown captured an additional 10 percent of 2013 MISO day-ahead congestion.

Overlapping Effect

Some of the aforementioned modeling changes overlap in terms of their congestion impact, i.e., different changes may affect congestion in a similar way and therefore one change will have less impact when the other changes are in place. The combined impact of modeling all the above categories of changes together resulted in a congestion level that is less than a straight sum of their individual impacts, and the difference is 21 percent of 2013 MISO day-ahead congestion.

Remaining Differences

This category represents the remaining difference between PROMOD simulation results and historical congestion, after all the aforementioned modeling changes were implemented. The remaining difference accounts for 24 percent of 2013 MISO day-ahead congestion, and it potentially stems from multiple sources such as market dispatch shift factor cutoff, day-ahead/real-time load variation modeling, loop flow representation, non-MISO area modeling and more. These potential causes cannot be tested due to the limitation of the simulation tool, scope of the study and finite amount of study time available. Currently, MISO is working with the vendor of PROMOD to implement some of the needed enhancements identified.

Summary of Results

The modeling changes yielded significant improvements in generation, LMP and especially in transmission system congestion.

Generation

The PROMOD Benchmark Study resulted in significant improvement in total generation (Figure 5.4-3). Specifically, coal and nuclear generation decreased and became closer to actual historical levels. Gas generation, particularly combined-cycle generation, increased to be closer to the actual historical level. As a result, the percentage of total generation and capacity factor by fuel type improved.

Figure 5.4-3: MISO total generation by fuel type

LMP

After all the aforementioned modeling changes, LMPs improved at all four commercial hubs in the MISO North/Central region, and became closer to their historical values (Figure 5.4-4). For example, differences between PROMOD results and historical market reduced from $9/MWh to $5/MWh for Illinois Hub and Minnesota Hub LMPs, and from $8/MWh to $1.5/MWh for Indiana and Michigan Hub LMPs. On a monthly basis, the LMP monthly pattern improved and differences between PROMOD results and the historical market decreased.

Figure 5.4-4: MISO North/Central commercial hub LMPs

Figure 5.4-4: MISO North/Central commercial hub LMPs

Transmission Congestion

The biggest improvement was achieved with transmission congestion. Congestion significantly improved across MISO North/Central region. When measured as the sum of the annual shadow prices, 76 percent of total 2013 MISO North/Central day-ahead congestion is captured (Figure 5.4-5). Monthly congestion pattern significantly improved as well. The congestion not only improved on an aggregated level, but also on an individual flowgate level. Namely, the distribution of the congestion across flowgates also improved significantly.

Figure 5.4-5: MISO congestion by total shadow price

Figure 5.4-5: MISO congestion by total shadow price

Conclusions

The PROMOD Benchmark Study was able to identify and quantify the impacts of many factors that led to the differences between PROMOD simulation results and actual market results. Among them, the biggest drivers are differences in transmission outages, generator outages and generator operating characteristics. After implementing the various modeling changes, the study replicated 76 percent of the 2013 MISO Day-ahead congestion, a nearly three-fold improvement in congestion from the Base PROMOD Model. Forty percent of this improvement may be applicable to future planning studies.

Identified best modeling practices will be vetted through stakeholder process, for instance, the Economic Planning User Group forum, before being applied in future planning studies. The modeling practices may include but are not limited to:

  • Generator Modeling Updates
  • Transmission Limit Adjustments
  • Non-conforming Load Modeling
  • Specific Phase Angle Regulator Modeling

The PROMOD Benchmark Study analyzes differences between the MISO market and PROMOD simulation tool, and identifies best modeling practices to enhance the accuracy and capability of the simulation tool in both backward- and forward-looking analyses.

The study started in 2014, and therefore 2013 was chosen as the most current focus year. A historical-looking PROMOD model, termed Base PROMOD Model, was built with 2013 data for load, gas prices, generation fleet and the transmission system. The simulation results of the Base PROMOD Model showed drastic differences with actual MISO market outcome, especially in regards to transmission system congestion and locational marginal price (LMP).

Significant efforts were spent identifying the causes of the differences and improving the model to minimize these differences. After applying various modeling changes, the Final PROMOD Model was able to capture 76 percent of the 2013 MISO Day-ahead Market congestion in terms of total shadow price, a nearly three-fold improvement in congestion from the 28 percent congestion captured in the Base PROMOD Model.

This substantial improvement came as a result of a number of modeling changes identified and implemented in the PROMOD Benchmark Study. To help understand the impact of these changes and identify the major contributing factors, they are classified into these categories:

  • Generation Outages
  • Generation Characteristic Changes
  • Transmission Outages
  • Transmission Derates
  • Other Modeling Improvements
  • Renewable Energy Updates
  • Pool Interchange Lockdown

 

Each category impacts MISO transmission system congestion differently (Figure 5.4-1). For example, the transmission outages category has the biggest impact at 18 percent, which means modeling transmission outages captured an incremental 18 percent of the 2013 MISO Day-ahead Market congestion. Other categories with significant impact are generation characteristic changes, generation outages, other modeling improvements and pool interchange lockdown.

Figure 5.4-1: Congestion impact of modeling changes in the benchmark study

Figure 5.4-1: Congestion impact of modeling changes in the benchmark study

The remaining difference of 24 percent can be attributed to various potential reasons, some of which may require enhancement of the simulation software itself.

Out of all the changes implemented in the PROMOD Benchmark Study, about 40 percent of the improvements are the result of modeling changes that may be applied to forward-looking analyses. The applicable modeling updates will be vetted through the stakeholder process before incorporation into future planning studies. These updates involve generator modeling, transmission limit adjustment, non-conforming load modeling and specific phase angle regulator modeling.

Study Process

The PROMOD Benchmark study process consisted of gathering historical information from both public and proprietary sources, analyzing the historical congestion pattern, comparing historical actuals with simulated results from PROMOD including generation and line flow, identifying discrepancies and potential causes, and eventually developing modeling changes and performing PROMOD simulations to verify the impact of the changes. Due to the complex nature of the issue, the differences seen between PROMOD simulation results and historical actuals usually stem from a multitude of causes rather than a single cause. If the simulation does not show enough improvements or shows unexpected results, additional information is collected, typically on a more granular level, to investigate the issue further and develop refined modeling changes for further testing. Therefore, the process is highly iterative (Figure 5.4-2).

Figure 5.4-2: Benchmark study process

Figure 5.4-2: Benchmark study process

Summary of Modeling Changes

The differences between PROMOD simulation results and MISO market actuals come from three different sources: input data accuracy and granularity; modeling approach and implementation; and inherent difference between PROMOD and the market. These differences manifest in many ways including market structure, simulation footprint, commitment and dispatch, modeling of generation, load, transmission, fuel, interchange and external areas. Each has varying levels of impact. The PROMOD Benchmark Study analyzed these differences in significant detail to identify modeling changes and needed enhancements to the PROMOD tool.

The various modeling changes implemented in the PROMOD Benchmark Study are categorized into a few groups (Figure 5.4-1), and each group of changes is elaborated on as follows:

Generation Outages

The PROMOD Benchmark Study modeled actual MISO North/Central region generator outages, including both planned and forced outages for 2013. Some generator outages in external areas were also modeled. The impact of these changes depended on the location of the generator outage relative to the constraint. Overall, by modeling the generation outages, an additional 9 percent of 2013 MISO day-ahead congestion was captured.

Generation Characteristic Changes

Using data from various sources, the operating characteristics of many generation units were modified, such as the unit’s heat rate, minimum capacity and maximum capacity. This change also included modifying the must-run statuses of various coal-fired and combined-cycle units for MISO and some neighboring areas based on historical data. This category of changes put generation output more in line with actual 2013 historical generation output. Overall, modifying these characteristics captured an additional 16 percent of 2013 MISO day-ahead congestion.

Transmission Outages

The study modeled the majority of its 2013 transmission outages, including both planned and forced outages, as well as some PJM 2013 outages. It should be noted that all transmission outages were modeled as planned outages in PROMOD due to its capability. Because powerflow can change significantly as a result of transmission outages, modeling these outages can have a dramatic effect on the congestion of specific flowgates. Overall, modeling of transmission outages captured an additional 18 percent of 2013 MISO day-ahead congestion.

Transmission Derates

Based on historical information, the study updated limits for flowgates in some focused areas. This generally increased congestion as it involved various rating decreases. It captured an additional 4 percent of 2013 MISO day-ahead congestion. The impact is more dramatic in the focused areas. For instance, the modeling of transmission derates captured an additional 22 percent of 2013 Northern Indiana Public Service Co.’s (NIPSCO) day-ahead congestion and an additional 13 percent of 2013 Ameren Illinois (AMIL) day-ahead congestion.

Other Modeling Improvements

This category of changes includes various modeling updates that do not fall under the rest of the categories. This includes non-conforming load modeling in some MISO and PJM areas, specific phase angle regulator modeling improvement and coal price update. Among these changes, non-conforming load modeling updates improved the distribution of congestion on flowgates. The impact of this category of changes is an additional 9 percent of 2013 MISO day-ahead congestion captured.

Renewable Energy Update

This change set the total amount of wind energy of MISO and PJM to actual wind energy for 2013. In the Base PROMOD Model, MISO had more wind energy modeled than historical, setting MISO wind energy to the actual historical amount tended to reduce congestion due to less west-to-east flow. At the same time, PJM had less wind energy modeled in the Base PROMOD Model than the actual historical amount, and therefore setting PJM wind energy to historical amounts increased congestion, particularly, in NIPSCO area. As a result of these changes, NIPSCO congestion increased by 11 percent of its 2013 day-ahead level, and AMIL congestion decreased by 8 percent of its 2013 day-ahead level. Overall, an additional 3 percent of 2013 MISO day-ahead congestion is captured.

Pool Interchange Lockdown

This change set the interchange between MISO and all neighboring pools at the actual historical interchange of 2013. This increased congestion on flowgates at or near the seams that were relevant to meeting these interchange schedules. Overall, modeling of the pool interchange lockdown captured an additional 10 percent of 2013 MISO day-ahead congestion.

Overlapping Effect

Some of the aforementioned modeling changes overlap in terms of their congestion impact, i.e., different changes may affect congestion in a similar way and therefore one change will have less impact when the other changes are in place. The combined impact of modeling all the above categories of changes together resulted in a congestion level that is less than a straight sum of their individual impacts, and the difference is 21 percent of 2013 MISO day-ahead congestion.

Remaining Differences

This category represents the remaining difference between PROMOD simulation results and historical congestion, after all the aforementioned modeling changes were implemented. The remaining difference accounts for 24 percent of 2013 MISO day-ahead congestion, and it potentially stems from multiple sources such as market dispatch shift factor cutoff, day-ahead/real-time load variation modeling, loop flow representation, non-MISO area modeling and more. These potential causes cannot be tested due to the limitation of the simulation tool, scope of the study and finite amount of study time available. Currently, MISO is working with the vendor of PROMOD to implement some of the needed enhancements identified.

Summary of Results

The modeling changes yielded significant improvements in generation, LMP and especially in transmission system congestion.

Generation

The PROMOD Benchmark Study resulted in significant improvement in total generation (Figure 5.4-3). Specifically, coal and nuclear generation decreased and became closer to actual historical levels. Gas generation, particularly combined-cycle generation, increased to be closer to the actual historical level. As a result, the percentage of total generation and capacity factor by fuel type improved.

Figure 5.4-3: MISO total generation by fuel type

LMP

After all the aforementioned modeling changes, LMPs improved at all four commercial hubs in the MISO North/Central region, and became closer to their historical values (Figure 5.4-4). For example, differences between PROMOD results and historical market reduced from $9/MWh to $5/MWh for Illinois Hub and Minnesota Hub LMPs, and from $8/MWh to $1.5/MWh for Indiana and Michigan Hub LMPs. On a monthly basis, the LMP monthly pattern improved and differences between PROMOD results and the historical market decreased.

Figure 5.4-4: MISO North/Central commercial hub LMPs

Figure 5.4-4: MISO North/Central commercial hub LMPs

Transmission Congestion

The biggest improvement was achieved with transmission congestion. Congestion significantly improved across MISO North/Central region. When measured as the sum of the annual shadow prices, 76 percent of total 2013 MISO North/Central day-ahead congestion is captured (Figure 5.4-5). Monthly congestion pattern significantly improved as well. The congestion not only improved on an aggregated level, but also on an individual flowgate level. Namely, the distribution of the congestion across flowgates also improved significantly.

Figure 5.4-5: MISO congestion by total shadow price

Figure 5.4-5: MISO congestion by total shadow price

Conclusions

The PROMOD Benchmark Study was able to identify and quantify the impacts of many factors that led to the differences between PROMOD simulation results and actual market results. Among them, the biggest drivers are differences in transmission outages, generator outages and generator operating characteristics. After implementing the various modeling changes, the study replicated 76 percent of the 2013 MISO Day-ahead congestion, a nearly three-fold improvement in congestion from the Base PROMOD Model. Forty percent of this improvement may be applicable to future planning studies.

Identified best modeling practices will be vetted through stakeholder process, for instance, the Economic Planning User Group forum, before being applied in future planning studies. The modeling practices may include but are not limited to:

  • Generator Modeling Updates
  • Transmission Limit Adjustments
  • Non-conforming Load Modeling
  • Specific Phase Angle Regulator Modeling