20  Module 3 - Day 3

Topics

For today’s practice make use of notebook module3-day3.ipynb created in your enviroment. Shut down kernel for all previous notebooks (if in runing condition) by right cliking on notbeook on left hand side file browser

20.1 HTTP protocol

Have a look at this url

https://finop-25.arcesium.vikrant.dev/live-notes/module3-day3.html

https: -> name of protocol .. use secured http for this transaction finop-25.arcesium.vikrant.dev -> host name.. name of the remote machine on which the server is running live-notes/module3-day3.html -> path of file on the server

HTTP gives us ways to manage remote resources on the webserver

There are four methods in HTTP:

  • get - gets the remote resouce as requsted by the client (firefox/chrome/program)
  • post - this allows to take some hidden parameter from client and according act on the server, if required it will also reponse back
  • put - put given resource on the server
  • delete - delete a resource from the server

20.2 Working with HTTP from python

We will use a third party library called requests for working with http functionality

pip install requests
import requests
url = "https://finop-25.arcesium.vikrant.dev/live-notes/module3-day3.html"
response  = requests.get(url)
response.status_code # status code 200 means the request was successful
200
#response.content # but this binary data
print(response.text[:1000])
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"><head>

<meta charset="utf-8">
<meta name="generator" content="quarto-1.8.26">

<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">


<title>20&nbsp; Module 3 - Day 3 – Arcesium Finop Python Training</title>
<style>
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
div.columns{display: flex; gap: min(4vw, 1.5em);}
div.column{flex: auto; overflow-x: auto;}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
ul.task-list{list-style: none;}
ul.task-list li input[type="checkbox"] {
  width: 0.8em;
  margin: 0 0.8em 0.2em -1em; /* quarto-specific, see https://github.com/quarto-dev/quarto-cli/issues/4556 */ 
  vertical-align: middle;
}
/* CSS for syntax highlighting */
html { -webkit-text-size-adjust: 100%; }
pre > code.sourceCode { white-space: pre; position: relative; }
pre > code.sourceCode > span { display: inline-block; line-height: 1.2
import requests

def download_data(url, outputfile):
    print("Downloading ...")
    r = requests.get(url)
    if r.status_code == 200:
        print(f"Writing to file {outputfile}")
        with open(outputfile, "wb") as f:
            f.write(r.content)
    else:
        print("There was an error in downloaing the file!")
download_data(url, "m3d3.html")
Downloading ...
Writing to file m3d3.html

problem - Write a function download_training_notes which will download all live notes of this training in use secified folder.

download_training_notes(baseurl, outputfolder)

hint: observer the url pattern

baseurl = "https://finop-25.arcesium.vikrant.dev/"
https://finop-25.arcesium.vikrant.dev/live-notes/module1-day1.html
https://finop-25.arcesium.vikrant.dev/live-notes/module1-day2.html
.
.
https://finop-25.arcesium.vikrant.dev/live-notes/module2-day1.html
.
.
import os
baseurl = "https://finop-25.arcesium.vikrant.dev/"
modules = 2
days = 5

def generate_urls_files(baseurl, modules, days):
    urls = {}
    
    for m in range(1, modules+1):
        for d in range(1, days+1):
            filename = f"module{m}-day{d}.html"
            urls[filename] = (f"{baseurl}/live-notes/{filename}")
    return urls

def download_training_notes(baseurl, outputfolder):
    if not os.path.exists(outputfolder):
        os.mkdir(outputfolder)
    else:
        print("Outputfolder already exits!")
        return

    urls = generate_urls_files(baseurl, 2, 5)
    for filename, url in urls.items():
        outputfile = os.path.join(outputfolder, filename)
        download_data(url, outputfile)
baseurl = "https://finop-25.arcesium.vikrant.dev/"
download_training_notes( baseurl, "downloaded_notes") 
Downloading ...
Writing to file downloaded_notes/module1-day1.html
Downloading ...
Writing to file downloaded_notes/module1-day2.html
Downloading ...
Writing to file downloaded_notes/module1-day3.html
Downloading ...
Writing to file downloaded_notes/module1-day4.html
Downloading ...
Writing to file downloaded_notes/module1-day5.html
Downloading ...
Writing to file downloaded_notes/module2-day1.html
Downloading ...
Writing to file downloaded_notes/module2-day2.html
Downloading ...
Writing to file downloaded_notes/module2-day3.html
Downloading ...
Writing to file downloaded_notes/module2-day4.html
Downloading ...
Writing to file downloaded_notes/module2-day5.html

index.html

https://finop-25.arcesium.vikrant.dev/ this means https://finop-25.arcesium.vikrant.dev/index.html

20.3 parameters of get request

google_search = "https://www.google.com/search?client=firefox-b-d&q=web+api&sei=VlSVadrbK4vp1e8P-ImC4Ag"
r = requests.get(google_search)
r.status_code
200
params = {"client":"python", 
                      "q":"web api",
                      "sei": "VlSVadrbK4vp1e"}
r = requests.get("https://www.google.com/search",
             params = params)
r.status_code
200

20.4 Web api

Web api documenation usually give type of request (get/post) and parameters for the get request. For example see the API documention of alphavantange

url = "https://www.alphavantage.co/query"
API_KEY= "UKVFE0JLE0TBPDEF" #"demo"
params = {
    "function": "TIME_SERIES_INTRADAY",
    "symbol": "IBM",
    "interval": "5min",
    "apikey": API_KEY
}
r = requests.get(url, params=params)
r.status_code
200
r.json()
{'Information': 'Thank you for using Alpha Vantage! This is a premium endpoint. You may subscribe to any of the premium plans at https://www.alphavantage.co/premium/ to instantly unlock all premium endpoints'}
r = requests.get("https://www.alphavantage.co/query?function=TIME_SERIES_INTRADAY&symbol=IBM&interval=5min&apikey=demo")
r.status_code
200
data = r.json() # it will actually convert the json data into python data
type(data)
dict
data.keys()
dict_keys(['Meta Data', 'Time Series (5min)'])
data['Meta Data']
{'1. Information': 'Intraday (5min) open, high, low, close prices and volume',
 '2. Symbol': 'IBM',
 '3. Last Refreshed': '2026-02-17 19:55:00',
 '4. Interval': '5min',
 '5. Output Size': 'Compact',
 '6. Time Zone': 'US/Eastern'}
timeseries = data['Time Series (5min)']
import pandas as pd
pd.DataFrame(timeseries)
2026-02-17 19:55:00 2026-02-17 19:50:00 2026-02-17 19:45:00 2026-02-17 19:40:00 2026-02-17 19:35:00 2026-02-17 19:30:00 2026-02-17 19:25:00 2026-02-17 19:20:00 2026-02-17 19:15:00 2026-02-17 19:05:00 ... 2026-02-17 12:20:00 2026-02-17 12:15:00 2026-02-17 12:10:00 2026-02-17 12:05:00 2026-02-17 12:00:00 2026-02-17 11:55:00 2026-02-17 11:50:00 2026-02-17 11:45:00 2026-02-17 11:40:00 2026-02-17 11:35:00
1. open 258.1100 258.1200 258.2342 258.2900 258.2900 258.0000 257.9858 258.1100 258.1100 258.2800 ... 258.9500 259.1400 259.3450 259.1501 258.7550 258.4500 258.2600 258.2800 258.3500 258.1600
2. high 258.2800 258.1200 258.2342 258.2900 258.2900 258.0000 258.1000 258.2198 258.1100 258.3100 ... 258.9800 259.4099 259.4100 259.5000 259.3000 258.7800 258.5099 258.4399 258.6900 258.6600
3. low 257.9300 257.9300 258.2198 258.2900 258.2100 258.0000 257.9858 258.1100 258.1100 258.2800 ... 258.4800 258.7700 258.7200 259.0000 258.7500 258.1950 258.1050 258.0800 258.1310 258.1300
4. close 258.2800 258.1100 258.2198 258.2900 258.2100 258.0000 258.0200 258.2198 258.1100 258.2900 ... 258.7800 258.9400 259.0850 259.3450 259.1100 258.7550 258.4500 258.1713 258.3450 258.3550
5. volume 107 52 5 1 11 1 167 5 13 1589 ... 21880 30102 29146 20922 40331 23839 21569 30208 34714 30303

5 rows × 100 columns

pd.DataFrame(timeseries).T
1. open 2. high 3. low 4. close 5. volume
2026-02-17 19:55:00 258.1100 258.2800 257.9300 258.2800 107
2026-02-17 19:50:00 258.1200 258.1200 257.9300 258.1100 52
2026-02-17 19:45:00 258.2342 258.2342 258.2198 258.2198 5
2026-02-17 19:40:00 258.2900 258.2900 258.2900 258.2900 1
2026-02-17 19:35:00 258.2900 258.2900 258.2100 258.2100 11
... ... ... ... ... ...
2026-02-17 11:55:00 258.4500 258.7800 258.1950 258.7550 23839
2026-02-17 11:50:00 258.2600 258.5099 258.1050 258.4500 21569
2026-02-17 11:45:00 258.2800 258.4399 258.0800 258.1713 30208
2026-02-17 11:40:00 258.3500 258.6900 258.1310 258.3450 34714
2026-02-17 11:35:00 258.1600 258.6600 258.1300 258.3550 30303

100 rows × 5 columns

following third party modules provide access to data from internet as python library

! pip install yfinance 
! pip install wbgapi

20.5 Authentication

Simple user password authentication

user = "dummy"
with open("/tmp/password.txt") as f:
    password = f.read()
r = requests.get("http://api.github.com/user", auth=(user, password))
r.status_code
401

20.6 json

json data is nothing but if you convert python lists/dictionary combined with basic datatypes in to text

import json
params
{'function': 'TIME_SERIES_INTRADAY',
 'symbol': 'IBM',
 'interval': '5min',
 'apikey': 'UKVFE0JLE0TBPDEF'}
json.dumps(params) # the way to convert python data into json string
'{"function": "TIME_SERIES_INTRADAY", "symbol": "IBM", "interval": "5min", "apikey": "UKVFE0JLE0TBPDEF"}'
pydata = {"value": [1, 2, 3, 4, 5],
          "symbol": ["X","Y","Z","A","B"]}
json.dumps(pydata)
'{"value": [1, 2, 3, 4, 5], "symbol": ["X", "Y", "Z", "A", "B"]}'
jsonstring = json.dumps(pydata)
jsonstring
'{"value": [1, 2, 3, 4, 5], "symbol": ["X", "Y", "Z", "A", "B"]}'
type(jsonstring)
str
d = json.loads(jsonstring)
d
{'value': [1, 2, 3, 4, 5], 'symbol': ['X', 'Y', 'Z', 'A', 'B']}
type(d)
dict

20.7 Library based example which works with data from internet

!pip install yfinance
Defaulting to user installation because normal site-packages is not writeable

Collecting yfinance

  Downloading yfinance-1.2.0-py2.py3-none-any.whl.metadata (6.1 kB)

Requirement already satisfied: pandas>=1.3.0 in /opt/tljh/user/lib/python3.12/site-packages (from yfinance) (2.3.3)

Requirement already satisfied: numpy>=1.16.5 in /opt/tljh/user/lib/python3.12/site-packages (from yfinance) (2.3.5)

Requirement already satisfied: requests>=2.31 in /opt/tljh/user/lib/python3.12/site-packages (from yfinance) (2.32.3)

Collecting multitasking>=0.0.7 (from yfinance)

  Downloading multitasking-0.0.12.tar.gz (19 kB)

  Preparing metadata (setup.py) ... done

Requirement already satisfied: platformdirs>=2.0.0 in /opt/tljh/user/lib/python3.12/site-packages (from yfinance) (4.3.6)

Requirement already satisfied: pytz>=2022.5 in /opt/tljh/user/lib/python3.12/site-packages (from yfinance) (2025.2)

Requirement already satisfied: frozendict>=2.3.4 in /opt/tljh/user/lib/python3.12/site-packages (from yfinance) (2.4.4)

Collecting peewee>=3.16.2 (from yfinance)

  Downloading peewee-3.19.0-py3-none-any.whl.metadata (7.0 kB)

Requirement already satisfied: beautifulsoup4>=4.11.1 in /opt/tljh/user/lib/python3.12/site-packages (from yfinance) (4.14.3)

Collecting curl_cffi<0.14,>=0.7 (from yfinance)

  Downloading curl_cffi-0.13.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (13 kB)

Collecting protobuf>=3.19.0 (from yfinance)

  Downloading protobuf-6.33.5-cp39-abi3-manylinux2014_x86_64.whl.metadata (593 bytes)

Collecting websockets>=13.0 (from yfinance)

  Downloading websockets-16.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.metadata (6.8 kB)

Requirement already satisfied: soupsieve>=1.6.1 in /opt/tljh/user/lib/python3.12/site-packages (from beautifulsoup4>=4.11.1->yfinance) (2.8)

Requirement already satisfied: typing-extensions>=4.0.0 in /opt/tljh/user/lib/python3.12/site-packages (from beautifulsoup4>=4.11.1->yfinance) (4.15.0)

Requirement already satisfied: cffi>=1.12.0 in /opt/tljh/user/lib/python3.12/site-packages (from curl_cffi<0.14,>=0.7->yfinance) (2.0.0)

Requirement already satisfied: certifi>=2024.2.2 in /opt/tljh/user/lib/python3.12/site-packages (from curl_cffi<0.14,>=0.7->yfinance) (2025.11.12)

Requirement already satisfied: python-dateutil>=2.8.2 in /opt/tljh/user/lib/python3.12/site-packages (from pandas>=1.3.0->yfinance) (2.9.0.post0)

Requirement already satisfied: tzdata>=2022.7 in /opt/tljh/user/lib/python3.12/site-packages (from pandas>=1.3.0->yfinance) (2025.2)

Requirement already satisfied: charset-normalizer<4,>=2 in /opt/tljh/user/lib/python3.12/site-packages (from requests>=2.31->yfinance) (3.3.2)

Requirement already satisfied: idna<4,>=2.5 in /opt/tljh/user/lib/python3.12/site-packages (from requests>=2.31->yfinance) (3.10)

Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/tljh/user/lib/python3.12/site-packages (from requests>=2.31->yfinance) (2.2.3)

Requirement already satisfied: pycparser in /opt/tljh/user/lib/python3.12/site-packages (from cffi>=1.12.0->curl_cffi<0.14,>=0.7->yfinance) (2.22)

Requirement already satisfied: six>=1.5 in /opt/tljh/user/lib/python3.12/site-packages (from python-dateutil>=2.8.2->pandas>=1.3.0->yfinance) (1.17.0)

Downloading yfinance-1.2.0-py2.py3-none-any.whl (130 kB)

Downloading curl_cffi-0.13.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB)

   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.3/8.3 MB 64.1 MB/s eta 0:00:00

Downloading peewee-3.19.0-py3-none-any.whl (411 kB)

Downloading protobuf-6.33.5-cp39-abi3-manylinux2014_x86_64.whl (323 kB)

Downloading websockets-16.0-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (184 kB)

Building wheels for collected packages: multitasking

  Building wheel for multitasking (setup.py) ... done

  Created wheel for multitasking: filename=multitasking-0.0.12-py3-none-any.whl size=15548 sha256=1d128fa11aafe308a1dfa192b727a107a115dd4de71a8a268a9f12f881a9e535

  Stored in directory: /home/jupyter-vikrant/.cache/pip/wheels/cc/bd/6f/664d62c99327abeef7d86489e6631cbf45b56fbf7ef1d6ef00

Successfully built multitasking

Installing collected packages: peewee, multitasking, websockets, protobuf, curl_cffi, yfinance

  WARNING: The script pwiz is installed in '/home/jupyter-vikrant/.local/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

  WARNING: The script websockets is installed in '/home/jupyter-vikrant/.local/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

  WARNING: The script sample is installed in '/home/jupyter-vikrant/.local/bin' which is not on PATH.

  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

Successfully installed curl_cffi-0.13.0 multitasking-0.0.12 peewee-3.19.0 protobuf-6.33.5 websockets-16.0 yfinance-1.2.0

import yfinance as yf
msft = yf.Ticker("MSFT")
import datetime
prices = msft.history(start=datetime.datetime(2021, 1, 1), end=datetime.datetime(2021, 12, 31))
prices
Open High Low Close Volume Dividends Stock Splits
Date
2021-01-04 00:00:00-05:00 213.526371 213.977356 206.118723 208.882202 37130100 0.0 0.0
2021-01-05 00:00:00-05:00 208.469590 209.678619 206.972710 209.083694 23823000 0.0 0.0
2021-01-06 00:00:00-05:00 203.585558 207.730777 203.364868 203.662323 35930700 0.0 0.0
2021-01-07 00:00:00-05:00 205.379873 210.465436 205.063237 209.457916 27694500 0.0 0.0
2021-01-08 00:00:00-05:00 209.832161 211.655296 208.248926 210.734131 22956200 0.0 0.0
... ... ... ... ... ... ... ...
2021-12-23 00:00:00-05:00 321.963188 325.485204 321.943847 323.840302 19617800 0.0 0.0
2021-12-27 00:00:00-05:00 324.585331 331.377781 324.556304 331.348755 19947000 0.0 0.0
2021-12-28 00:00:00-05:00 332.026116 332.664724 329.287868 330.187714 15661500 0.0 0.0
2021-12-29 00:00:00-05:00 330.236038 333.138786 328.668558 330.864990 15042000 0.0 0.0
2021-12-30 00:00:00-05:00 330.826257 332.006709 327.836430 328.320221 15994500 0.0 0.0

251 rows × 7 columns