전력수급 실시간 crawling

http://epsis.kpx.or.kr/epsisnew/

 

전력수급 > 실시간 전력수급 데이터 화면(아래)를 크롤링

 

from selenium import webdriver
from bs4 import BeautifulSoup
import numpy as np
import pandas as pd

driver = webdriver.Chrome("./chromedriver")
#3초 기다려주기, 웹페이지 로딩까지
driver.implicitly_wait(3)  
driver.get("http://epsis.kpx.or.kr/epsisnew/selectEkgeEpsMepRealChart.do?menuId=030300")
soup = BeautifulSoup(driver.page_source, 'html.parser')

a = []
for i in soup.find("div", class_='tablewrap').find_all("td"):
    print(i.text)
a

idx = [] 
for i in soup.find("div", class_='tablewrap').find_all("th")[6:]:
    idx.append(i.text)
idx

columns = ['공급능력(MW)','현재부하(MW)', '공급예비력(MW)','공급예비율(%)','기온(℃)']
df = pd.DataFrame(np.array(a).reshape(-1,5), columns = columns, index = idx)
df

 

20분 뒤 다시 실행

driver = webdriver.Chrome("./chromedriver")
driver.implicitly_wait(3)  
driver.get("http://epsis.kpx.or.kr/epsisnew/selectEkgeEpsMepRealChart.do?menuId=030300")
soup = BeautifulSoup(driver.page_source, 'html.parser')
a = []
for i in soup.find("div", class_='tablewrap').find_all("td"):
    a.append(i.text)
idx = [] 
for i in soup.find("div", class_='tablewrap').find_all("th")[6:]:
    idx.append(i.text)
columns = ['공급능력(MW)','현재부하(MW)', '공급예비력(MW)','공급예비율(%)','기온(℃)']
df_new = pd.DataFrame(np.array(a).reshape(-1,5), columns = columns, index = idx)
df_new

 

 

df = pd.concat([df_new, df_1])
df

2021-12-19 18:50 중복으로 중복제거 필요

 

df = df.loc[~df.index.duplicated(keep='first')]
df