DEV Community: technonotes-hacker

Core Python - Beginner & List Intro

technonotes-hacker — Thu, 22 Oct 2026 03:47:00 +0000

Python

English to Code
Interpreter Language --> Line by line execution
In Java --> First it will create Binary file which is nothing but .CLASS File and this CLASS file will execute.
In python hello.py --> python hello.py ( Execution )

Data Structure

How do you store the data ?
List , Tuple , Dictionary and Set
The value will be stored in the RAM.
Identifier is like a LABEL.
The Value is stored in the RAM , but to REFER the place in the RAM is called the name. i.e the LABEL --> in the below example.

Boolean

Let it be like a Question related.

None

print(None)

Continue ...

In Python all are Object.
What is an Object ? --> Eg., Pen is an Object.
Object will have 2 things:

Attribute --> Ink or ball point or brand
Functionality --> what you are going to do with this Pen ? or Action.

How to know about the Object ?

For the string , these many objects are there.

Data Structures

List

Its a Array
Its a Dynamic Array
Consecutive storage.
In Array , it will be fixed . That means the size , also same datatypes. But in Python we have Dynamic Array.
No Fixed Size and any data type.

Negative index is also available.

Square bracket and position is important.

Going to upend a Alphabet.

In first position , upend "G". ( Its like moving and inserting the alphabet ) . It won't delete anything .

Even you can replace --> vowels[2] = "H"
Store all datatypes.

LLM - RAG - Implementation

technonotes-hacker — Tue, 20 Oct 2026 15:54:35 +0000

What you need first ?

Load PDF
Split PDF --> The output is Chunks.
Build Vector DB ( Input will be CHUNKS ) --> Also need instance or object to Query.
Retriever
Augment or Build Prompt
Generate_answer --> ask the LLM --> Groq
RAQ Query

Things Needed

Embedding --> nomic-embed-text ( Model )

Install Ollama

>llama --version
ollama version is 0.40.6

> ollama list
> ollama pull nomic-embed-text

Test with the below curl URL if you are getting the response.

what is this nomic-embed-text Model --> Its just a Weight , to execute this we have some tools like Ollama , lama.cpp , etc

Another part of LLM

LLM for retrieval via Groq ( Chatgpt )

2 LLM

1 local embedding model
Another 1 , LLM model via Groq

Lets Begin

Load the Env details

Env variable

Load the PDF

PyPDFLoader
This loader will take care all the things.

Set the env and run requirement txt

python3 -m venv .venv
source .venv/bin/activate
(.venv) pip install -r requirements.txt
pip freeze > requirements.txt
python3 -m pip install pypdf

Run and check the PDF Part

This code will read the PDF and give the details.
It converts the entire PDF into text.

You can see many Chinese characters , this needs to be cleaned and sent for Chunking.

Clean the docs

Write one more function to clean the documents.
The complete document is in list. It will be like each page.

def clean_docs(documents):
    """Normalizes whitespace, removes null bytes, and sanitizes text encoding for each document."""
    cleaned_docs = []
    for doc in documents:
        # Handle objects with page_content attributes vs raw text strings
        if hasattr(doc, "page_content"):
            text = doc.page_content
        else:
            text = str(doc)

        # Replace multiple whitespace characters with a single space
        text = re.sub(r"\s+", " ", text)
        # Remove null control characters
        text = re.sub(r"\x00", "", text)
        # Drop invalid UTF-8 sequences and decode back to a clean string
        text = text.encode("utf-8", "ignore").decode("utf-8")
        text = text.strip()

        # Update the document object or append the cleaned text
        if hasattr(doc, "page_content"):
            doc.page_content = text
            cleaned_docs.append(doc)
        else:
            cleaned_docs.append(text)
    return cleaned_docs

Check for the 100th page

Comment the line and check if getting after the cleaning

Change the function like below

Now we can read the PDF. i.e load the PDF got completed.

Chunking Part

function split , its a text splitter.

The above one is a CHUNK
Now another CHUNK - Its in yellow color. ( with a overlap )
" Such as ... " is a overlap. , it will be both the CHUNK.
Overlap means each chunk repeats the last 150 characters of the one before it. If a sentence is cut in half at the edge, it still appears whole in one of the chunks.

This is called - Recursive character text splitter, langchain already implemented this.

from langchain_text_splitters import RecursiveCharacterTextSplitter

Need to provide the CHUNK size and also the overlap.
If the PDF as diagram this code won't work.

def split_pdf(documents):
    """Splits documents into smaller overlapping chunks to optimize for embedding and retrieval."""
    splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=150)
    return splitter.split_documents(documents)

Now vector DB

The input to this is " CHUNKS ".

Embeddings
For embedding , we are going to use Ollama Model which is in the local. For this Ollama model , langchain is supporting.

from langchain_ollama import OllamaEmbeddings

Persist directory is used.
Return the Vector Store.

DB's

Sqlite3 --> Its a Document DB --> No need to install like Postrgres , its just a python package . ( DB browser for SQLITE )
In-memory DB's. [ TBD ]

Initial Setup

def initial_setup(pdf_path):
    """Executes the end-to-end ingestion pipeline: loads, cleans, chunks, and vectorizes the PDF."""
    chunks = load_and_clean_pdf(pdf_path)
    vectorstore = build_vectorstore(chunks)
    return vectorstore

Groq LLM

gpt-oss-120b
User query given to this LLM.
Get the response from the LLM for the user query.

Retriever

Sending the Query to the embedding.
[ TBD ]

Build the Prompt

[ TBD ]

Notes

We need some tool to learn the Model , those tools are called Ollama , lama.cpp , etc
If you want to run Python script , what you need ? Python right ? Like wise we need Ollama tool to run the Model.
Model is just like a file.
Zed( Editor ) is like Windows VS Code Editor
Langchain is used to create the Agent , its like a single Framework.
If its scanned PDF then reorganization model needs to be used then also need to use OCR.
White space regular expression.
Import re --> regular expression.
Unicode - utf8 --> Its a text format or Unicode format.
Langchain is like Broker , so it supports OpenAI , Claude , etc .
Langchain as many contacts so its a broker.
Even Chroma is included in the Langchain ECO-SYSTEM. Otherwise we need to install and write program for integration of Chroma.
Langchain is a wrapper.
Langchain satisfies the end user , you can buy everything here. That the idea of Langchain.
Langchain & Chroma DB is a open source.
Chroma DB in cloud is Paiza.
PineCone / PG Vector ---> [ TBD ]
SQL Query will not work in Vector DB.
What the retriever.invoke is doing ? [TBD]
The input of the PDF is SOFTWARE ENGINEERING.
Recursive means it tries to cut at nice places first: paragraphs, then lines, then sentences, and only cuts in the middle of a word as a last resort.

Error

Surrogates not allowed

Questions

RAG ----> Retrieval-Augmented Generation: find the relevant paragraphs first, then answer using only those.

Why RAG? ----> The model hasn't read your documents. RAG is cheaper and fresher than pasting everything or retraining.

Indexing vs querying ----> Indexing: prepare the document once. Querying: for each question, find chunks and answer.

Document loader ----> Reads a file into Documents, e.g. PyPDFLoader gives one Document per page with page_content and metadata.

Chunk ----> A piece of text (about 1,000 characters) small enough to search and send to the model.

chunk_overlap ----> Repeated characters between neighbouring chunks, so sentences cut at an edge are still whole in one chunk.

RecursiveCharacterTextSplitter ----> Cuts at nice places first (paragraphs, lines, sentences) and mid-word only as a last resort.

Embedding ----> A list of numbers for a text's meaning. Similar meaning = close together.

Vector database ----> Stores chunks with their embeddings and finds the closest ones to a question (Chroma, here).

Retriever and k ---> Finds the best chunks for a question. k = how many chunks to return.

RAG prompt ----> "Answer using ONLY the context. If it's not there, say you don't know." plus context and question.

Show sources ---> Each chunk has its page in metadata. Show it so readers can check the answer.

Terms - AI ( PhD )

technonotes-hacker — Tue, 20 Oct 2026 11:01:48 +0000

LangChain

Programming Framework
LangChain is an open-source orchestration framework designed to help developers build applications using large language models (LLMs)
Its used by developers to connect LLMs to external databases, APIs, and tools so they can become "Agents."
Who is the Agent here ? --> No, LangChain itself is not the agent..
LangChain is the Operating System / Framework .
The Agent is the running system --> "Agent" is the final, fully functional AI system.

LangChain is an LLM framework.

Generative AI (GenAI)

Generative AI is a tool that creates text, code, or answers based only on what you ask it to type. It cannot take action, it cannot browse live websites on its own, and it cannot click buttons. It just talks.

Generative AI is about Thinking & Writing.
Input Prompt → Output Text.
RAG is a system design used within Generative AI.

AI Agent / Agentic AI

The Autonomous Assistant.
An AI Agent is given a final goal.
Instead of just typing an answer, it has access to tools (like the internet, a calculator, or software applications).
It can think in a loop: it takes a step, looks at the result, decides what to do next, fixes its own mistakes, and doesn't stop until the goal is fully achieved.

Agentic AI (Agents) is about Thinking, Deciding, & Taking Action.
Input Goal → Loops through Tools → Achieves final result.

Ollama

The software that manages and runs the LLaMA and Qwen AI models on your computer or server.

Overall Structure

Ollama and LangChain are completely different layers of the software stack.

Project Statement

The project aims to build a generative Artificial Intelligence (AI) powered Large Language Model (LLM) using Retrieval Augment Generation (RAG) technique to provide responses to user queries in ABC portal related to Master Circulars, e-Circulars and Role Based Certification (RBC) manuals as the knowledge base for the GenAI model.

Lets Dig one by one

LLM - Data Validation with Pydantic

technonotes-hacker — Tue, 20 Oct 2026 09:40:12 +0000

While typing a prompt to LLM and asking the LLM to Impersonating a USER --> what is the prompt called ? ROLE PROMPTING
ROLE PROMPTING is similar to which Prompting --> SYSTEM Prompting.
Main difference --> Boundaries are defined , System Prompting --> As a AI agent , ROLE Prompting --> As a Bank Manager.

Is the output is Structured ?

structured or Text Manner ?
By default it won't come , we need to define. How the output should be.

RAG System

What is Augment ? What will happen ? --> It happens between Retrieval and generation.

DATA VALIDATION

Python is Dynamic Typing.
Output of LLM , think like text.
Get the output in the desired format.
There is package called "Pydantic" , data validation.
Pydantic is a library in Python .
We need to learn --> Live with it --> Data Validation & Data Modelling.
Data that comes into your program, from a form, a website or an AI model, can be wrong: a missing field, text where a number should be, an age of -9. Validation means checking data before you use it.
Pydantic lets you describe the shape you expect once, as a class, and then checks everything for you.

List

List is a immutable element.

Agent

Predefined flow is not available in Agent.
How to pass the data ?

Questions

Validation --> Checking data before using it, at the gate rather than on the plane.

Pydantic BaseModel --> Describe the shape you expect as a class; Pydantic checks every value against it.

ValidationError --> Lists every problem at once: which field, which rule and which value.

gt / ge / lt / le --> Greater than / greater or equal / less than / less or equal rules for numbers.

min_length / max_length --> How long text (or a list) may be.

Optional field --> bio: str | None = None. May be missing.

default_factory=list --> Makes a fresh empty list every time instead of sharing one.

model_dump() / model_dump_json() --> Turn a model into a dict / into JSON text.

@field_validator --> Your own rule for a field. Raise ValueError to reject; always return the value.

Nested models --> A model inside a model (Order > Customer > Address), all checked with one call.

FastAPI + Pydantic --> Bad request = automatic 422. response_model filters what you send back (no password leaks).

with_structured_output(Model) --> Gets a validated Python object from an LLM instead of text you have to parse.

Notes

https://pydantic.dev/docs/
JSON Validator

LLM - Tool Calling & MCP

technonotes-hacker — Tue, 20 Oct 2026 06:54:38 +0000

Function Calling - The Manual Tool LOOP

LLM ( ChatGroq ) Will say which tool needs to be called.
List or the Object will have some value.

This is with Langchain and one type of Tool Calling.

Custom Tool

Its a structured way.

External API Tools

These are the APIs which we are going to call.

Database Tools

Create a custom table.
We are allowing only SELECT in PROD.
But if we use HUMAN in the loop , then update can be also used - But make sure its also RISK to use in PROD to update a query.
Qwen Models --> Mostly you can use this for local testing.

MCP - Model Context Protocol

Its a protocol.

MCP is a common standard (like USB for AI tools).
MCP client will decide which method to call.

இப்ப ஒரு API expose பண்ணும்போது அதுல PUT GET எல்லாமே இருக்கும், but இந்த MCPல , APIல எல்லாத்தையும் club பண்ணி , எல்லாத்தையும் expose பண்ணுவாங்க . MCP cleint will take care which one to call like a Agent.

Questions & Things to know

Notes

Langchain is a Framework.
Function description --> https://developers.openai.com/api/docs/guides/function-calling?api-mode=responses
Tool Calling ( description , above one )
With pydantic we achieved DATA validation --> Return you will get in Object flow.
Sqlite --> File based DB.
esh --> english to shell , project --> https://github.com/numericalworks/esh
ER Model Will be generated from the tool , before LLM become famous.
Ollama or Llama cpp ?
Custom Tool or Function calling --> Both are same , there are different methods.
General Method --> API with swagger , You will get the request and response with PUT GET etc. Now the MCP Client will search and decide , which method to call.

LLM - Memory for Agent

technonotes-hacker — Sun, 18 Oct 2026 10:15:44 +0000

Memory Systems

End Goal is to create AI agent.
Agent is like a orchestrator + Integration of LLM + Action of Another tool + Memory.
We need to give MEMORY to the Agent.
Memory means --> இதுக்கு முன்னாடி நான் இது கிட்ட என்னென்ன கேட்டு இருக்கேன் & அது கொடுத்த விடை. [ Query & Response/Answer ],--> This combination is going to sit in the MEMORY.

How to store ? We cant't store continuously.
What is the LIMIT ?
There are types , lets discuss one by one.

List of Memory

1. Buffer 
2. Window
3. Token
4. Summary
5. Long Term
6. Semantic

BUFFER MEMORY

For each chat , we are going to store complete conversation, like below.

In-memory --> RAM
Persistent Storage --> like --> SQLITE3 , POSTGRES

For each and every call , whole conversation history is going to be sent to LLM.
For new chat , its going to have a DIFFERENT IDENTIFIER.
What problem can be identified in these 3 above points ? ஒரு சின்ன பிரச்சனை இருக்கு , அது என்ன ?
Point 2 --> We need to send the complete HISTORY to the LLM --> Tokens gets affected , also CONTEXT Limit gets affected. But you will get the output.
In the code , InMemorySaver() --> Will help to store in the RAM Memory.
With the capacity of RAM storage only it can store.

WINDOW MEMORY

It keeps only a FIXED NUMBER OF CHATS / MESSAGES. Eg., First message + Last N messages.
In Langchain --> There is a provision like a MIDDLE-WARE --> Modelகு முன்னாடி , இந்த Middlewareஅ தாண்டி தான் போகணும் .
In Middleware we can configure --> Remove message , just save the first & last messages alone.
Request goes to Middleware and then only MODEL.

What will pass in State and Runtime ?
Reply will be also taken along with the question.

Now remove all messages and stores only 5 messages.

This memory scenario is used only when recent conversation are important then OLD/PREVIOUS conversation.

TOKEN MEMORY

Similar to WINDOW memory.
Will set the tokens. Eg., 2000 Tokens .
Not going to store more than 2000 Tokens .

Token definition will defer from Model to Model and framework to framework.
Eg., 1000 tokens --> each character is a token in the above text.
Old data's will be removed.

arised ……………………….Ipsum → இவ்ளோ words இந்த paragraph ஆயிரம் டோக்கன் வெச்சுக்குங்க ( 1000 tokens ) , but popularised word is broken , பாதியிலிருந்து அந்த word வந்து எடுத்துக்குது , நம்ம இதுல set பண்ணலாம் , பாதி word வெச்சிக்கிட்டு நம்ம என்ன பண்ண முடியும் . Even though 1000 tokens are used , to get the proper meaning , it will take ( that means we will set ) , take from the start of the sentence. “ It was popularised …… Ipsum .” --> This will be handled by the Human conversation message.
trim_messages() --> Function will do the honour in this.
In scenarios --> It will be useful only for recent messages. OLD messages will be deleted as per the token size.
Anything older than the given budget is lost completely. --> this is the DOWNSIDE.

SUMMARY MEMORY

Human & AI → இதுல AI response பண்ணுது and Human கேள்வி கேக்குறாங்க. இப்ப இதுல , memoryல 4 conversation தான் store பண்ண முடியும் but first ரெண்டு போயிடும் . அதுக்கு முன்னாடி , ஒரு summary மாதிரி நம்ம save பண்ணி வச்சுக்கலாம் . So that , even the data gets deleted but we will have the summary. This summary can be created by passing to a cheap LLM for cost effectiveness.
For collating the summary --> Use Cheap Model.
SummarizationMiddleware () --> This function will be used.

LONG TERM MEMORY

Agent will decide were to store and what to store .

SEMANTIC MEMORY

Entire conversation into a POINT.
Its like in RAG.
Store in Vector DB.

Questions & Synopsis

Model needs memory ? --> They remember nothing: every call is fresh. The app must resend earlier messages.

Buffer memory --> Keep every message. Simple, but it grows forever (cost and size limit).

Window Memory --> Keep the first message plus the last few. Forgets the middle.

Token budget --> Keep the newest messages that fit a token limit (trim_messages). Still forgets old things.

Summary memory --> Replace old messages with a short summary so facts survive. Costs an extra AI call.

Long-term memory --> Facts saved in a Store, shared across a user's conversations. The agent decides what to save.

Semantic memory --> Long-term facts searched by meaning with embeddings. Needs an embedding model.

Checkpointer --> Saves the conversation after each turn (InMemorySaver; SqliteSaver or a database for persistence).

thread_id --> The conversation's file number. Same id continues it; a new id starts fresh.

Middleware (@before_model) --> A function that runs right before each AI call and can edit the message list.

Store vs checkpointer --> Checkpointer = one conversation. Store = lasting facts about the user across conversations.

user_id in memory tools --> Comes from your app's context, not from the AI, so users can't read each other's memories.

Notes

What is In-memory --> its RAM.
What is Langchain ?
What is decorator ?
what is lorem ipsum ?
What is Key word ?

Promotion - Oracle ASM

technonotes-hacker — Sun, 11 Oct 2026 15:59:04 +0000

Oracle Automatic Storage Management (ASM) is a built-in volume manager and file system for Oracle Databases that simplifies storage administration.
Its a Volume Manager
What will the Volume Manager will do ? and what do you mean by Volume ?

5TB Big Data , you need to store this --> In storage every single disc is having only 1TB. 
5 Hard Disc Needed. 
Multiple Disc into single disc to the outside world.
1 Volume can have 1 Physical Hard Disc or Multiple Hard Disc.

These storage software will take care the Volume Manager also.
If we assign Multiple disc to ASM , ASM can handle the disc , ASM add the disc from the Volume , Remove the disc from the volume , ASM can perform the redundancy of the storage, ASM can perform the Load Balancing inside the disc.
ASM provides the Volume Manager layer which is needed by every storage software.
ASM also acts as a File System.

What you mean by File System ?

Its a storage area.
File System is basically a STORAGE.
File System as Storage area for Oracle Database.

Oracle Clusterware

You can use it for Cluster Management not only for Oracle but for any other infrastructure.
It can be installed to manage the cluster , it doesnt mean that you need to have only Oracle RAC.

Oracle ASM --> its is File System Storage only for Oracle RAC / Standalone Database. Its not supported for others.

What type of File can store in ASM ?

RMAN Backup
Flash Back Log
Export Import Files
Expect PFILE , all are in Binary Mode. So all files can be stored in ASM.

DiskGroups in ASM

Collection of disks which make a volume ( diskgroups )
Diskgroup is nothing but Volume.
Diskgroups --> Collection of disk at the OS level.
You can assign / must assign a name to diskgroup. Eg., ASM1 /dev/sdc1 --> The ASM1 name will be assiged to the physical disc.
Minimum ASM Structure defined by Oracle is --> 3 Discgroups for basic Oracle RAC configuration.

CRS Diskgroups --> To store cluster related files
DATA Diskgroups --> To store database files like datafiles etc
FRA Diskgroups --> To store FRA files

FRA --> Fast Recovery Area

Where we named ASM1 ? At what place ?

When we installed the ASM Binary , Post ASM binary , we had issued a command " Oracle ASM CREATE DISC ".

What is DISkS ?

Physical HDD Storage
One HDD can't belong to two discgroups
But Multiple HDD can be belonged to one single discgroup.
Types of storage considered for ASM:

Complete HDD partition.
Multiple HDD Partitions assigned to single ASM disks
NFS
LUNs allocated from Storage. Oracle doesn't recommend this ? Why ?

ASM is already a Volume Manager, so when allocate LUNs from the storage server , storage software is again a Volume Manager. Two Volume Manager are not recommended for storage of data. Because you will have Redundancy of Volume Manager.

Eg., for point 2

/dev/sdc
/dev/sdc1 --> ASM2 ( These are only disk )
/dev/sdc2 --> ASM3
/dev/sdc8 --> ASM7

ASM2 is a single disk , can be assigned to one diskgroup. But ASM2 can't be assigned to Multiple Diskgroups.

1 single diskgroup can have multiple disk , it means , single diskgroup can have ASM2 , ASM3 .
For ASM1 , I can add --> ASM1 /dev/sdc1 + /dev/sdd1 + ..
These disk will have datafile.

Lets get inside the ASM Instance

What is Oracle ASM Instance on STANDALONE DATABASE?

Its built on same technology as of database instance.
SGA + BG Processes ( BackGround )
No Redo Logs
Its just a mediator between Oracle Instance and the Storage Disc.
Its very small when compared to Oracle Database Instance as it performs very small task.
Its smaller in size when compared to DB instance.
where is Oracle ASM installed ? Oracle ASM is installed in GRID_HOME. ASM binary are installed under GRID HOME.
What exactly stored ? what is the metadata inside the ASM ? --> ASM manages the exact metadata of the disk group and provide file layout to Oracle Instance.
ASM instance is like a mediator between Storage ( diskgroup ) and Oracle Instance.
Every single machine as one ASM instance running on it.
3 DB instance on a single machine ==> 1 ASM instance shared by all the DB instances.

What is Oracle ASM Instance on RAC DATABASE?

Every single machine / node has one ASM instance running on it.

Oracle ASM Instance Failure

If the Oracle ASM fails on one node --> what will happen ? --> what happens to the Oracle DB instance ? or If the Oracle Database is running and suddenly Datafiles are deleted , what will happen ? DB instance FAILS. Same DB Instance also fails if ASM fails. ( DB instance will also FAIL )
Can I stop the ASM instance when DB is up and running ? ( Important Interview Question ) ---> Stop all the DBs instance accessing the ASM before stopping the ASM instances. This is the procedure.
From 12c RAC --> ASM Instance Fail over was introduced --> Oracle ASM Flex Cluster , need to be enabled.
what the ASM flex Cluster will do ? --> If the ASM in node 1 fails , then it communicates to ASM in node 2 and then access the shared location.

This is only available only from Oracle RAC 12c.
This is like High Availability.

ASM Mirroring & Failure Group

Mirroring is like Maintaining a copy of your data.
Its like "A diskgroup is mirrored to another diskgroup".
Oracle ASM itself provides the redunduncy.
You have to mention the redundancy at the time of disk group creation.
Once done , it can't be changed.
Types of Redundancy in ASM :

Normal --> 2 way redundancy --> 2 failure groups , Max you can have 3 Failure groups.
High --> 3 way redundancy --> 3 failure groups , Max you can have 5 Failure groups.
External --> No ASM level redundancy or you can have external Softwares for redundancy.

What is Failure Groups ?

One Single Failure Group is one Single redundant Copy.
To increase the redundancy , we can have multiple Failure groups.

Failure Group are defined while creating the disk group.
Failure Group is nothing but the redunduncy , no of failure group is nothing but the level of redunduncy.

ASM Adminstrator

ASM needs the Utility to administor.
SQLPLus ( How ? because ASM instance is like Oracle DB instance , so SQLPlus can be used )
ASMCMD --> ASM command mode
ASMCA --> ASM configuration ASSISTANCE --> its like GUI ( All things can be performed , even the upgrade of ASM ).
These are the Utility used. But these are only for STANDALONE DB.
What about RAC based Oracle Instance ? For Multiple ASM , then SRVCTL --> Clusterware Level Utility which allows you to administer all ASM Resources.

Note

DiskGroups as Table space
Disks --> is a Datafile.
In a tablespace can have multiple datafile. But single datafile can't be assigned to two tablespace.
SGA --> System Global Area
CRS --> Cluster Ready Services
LUNs --> Logical Unit Numbers
Database backup is taken by RMAN. ASM will allocate the channel to the diskgroup. No ASM comes inbetween for backup . RMAN directly write into the ASM diskgroup. No over head on the ASM instance.
When you take RMAN backup for OS level disk , does it give over head to the OS system ? Not really . Same , if you allocate channels to ASM diskgroup , RMAN will continur to write and ASM instance continue to perform until the diskgroup is full or ...
If you talk about Volume Manager --> Stripping , Mirroring , Load Balancing --> Everything is supported by Oracle ASM.
Irrespect of Database instance in a single node , only one ASM will be running for each node. That is the benefit .
Mostly EXTERNAL will be configured , as they will allow the storage team at HARD DISK level.
SRVCTL --> Server Control

Promotion - CertIn

technonotes-hacker — Sun, 11 Oct 2026 10:55:13 +0000

Indian Computer Emergency Response Team ( CertIn )
Its pivotal organization formed by Ministry of Electronics and Information Technology 2004.
Effective response to Cyber Incidents.

Role

Safeguard systems and network from:

Cyber threats
Including Malware , Hacking and Data Breaches

Its key role is to monitor and analyse cyber threats and Vulnerabilities.
Source : Government Agencies , Private Sector Organization , International Partners which helps to identify emerging threats and trends.
The advisories and alerts are circulated amoung stakeholders.
It also conducts awareness program.
Fostering a culture of Cyber Security AWARENESS.
Aims to empower individuals and Oraganization to protect themselves against Cyber threats.
The Role of CERT-In in ensuring cyber security is becoming more critical day-by-day.

Technical Guidelines on | SBOM | QBOM & CBOM | AIBOM | HBOM |

SBOM

Software Bill of Materials
A complete inventory of software components used in an application.
Includes:

Libraries (open source + proprietary)
Versions
Licenses
Dependencies
Vulnerability mapping

AIBOM

AI Bill of Materials.
It includes servers, sensors, and GPUs. Also AI models , frameworks ,and development tools.
Inventory of all AI‑model components, mandatory for AI systems.

Model name, type, version
Training dataset sources
Algorithms & architecture (LLM, CNN, etc.)
Open‑source AI components
External integrations
Governance approvals (MANDATORY per CERT‑In)

Benefits of AIBOM

Minimum Elements Required

Recommendation & Best Practices

The format for generating AIBOM for AI solutions should adhere to established standards, such as Software Package Data eXchange (SPDX) or CycloneDX, ensuring compatibility and uniformity across the industry.
The AI developer/integrator organization that supplies AI solutions to government and public sector organizations should develop a Vulnerability Exploitability eXchange (VEX) for AI models after a vulnerability or security issue is discovered.

HBOM

Hardware Bill of Materials
Inventory of hardware components supporting the application.

Servers
Network devices
Storage systems
Firmware versions
End‑of‑life/End‑of‑support status

CBOM

Configuration Bill of Materials
Inventory of configuration elements that define how the system is deployed.

Config files
Environment variables
Parameter sets
Network config (ports, protocols, IPs)
Deployment descriptors

QBOM

Cloud Bill of Materials
QBOM is referenced alongside CBOM in the CERT‑In document.
A structured inventory of all cloud components used by an application.

Cloud services (AWS, Azure, GCP, etc.)
Compute resources (VMs, containers)
Storage buckets / databases
Identity & access configurations
External APIs hosted on cloud

Notes

Pivotal --> extremely important or central because everything else depends on it.
foster --> to help or encourage the development of something (especially feelings or ideas).

Kubernetes - Day - 09 - Docker Quiz

technonotes-hacker — Sun, 11 Oct 2026 10:05:05 +0000

Just for first time --> docker run hello-world --> Docker can't find the image locally, so it pulls it from Docker Hub --> With no image on your machine, docker run pulls it from the registry, then creates and starts the container.

time docker run --rm alpine echo "hi" --> Run this command twice --> the second run much faster? --> The image was already downloaded on the first run. --> The first run includes pulling the alpine image. After that, starting the container is just starting a process.

docker run --rm alpine echo hi --> Removes the container automatically when it exits.

Which registry does Docker use when you don't name one? --> Docker Hub (docker.io)

Which command lists images on your machine? --> docker images

What problem does Docker mainly solve? --> "It works on my machine": packaging an app with everything it needs so it runs the same everywhere --> An image carries the app plus its libraries and settings, so dev, test and production run the same thing.

What do all containers on one host share? --> The host's Linux kernel. --> Containers are isolated processes on the host. Each gets its own view of files and processes, but they all use the host's kernel.

Why does a VM usually take longer to start than a container? --> A VM boots a complete operating system; a container just starts a process. --> A VM is a whole pretend computer with its own kernel. A container is a normal process with a private view.

Which program lets one physical server run several virtual machines? --> A hypervisor (KVM, VMware, VirtualBox)

An image is to a container as… --> a recipe is to a cooked dish (or a class to an object) --> One image can start many containers, just like one recipe can feed many plates.

Why does Docker Desktop run a small Linux VM on Windows and macOS? --> Linux containers need a Linux kernel, and Windows/macOS don't provide one.

Which statements about containers are true? (Pick all that apply.) --> They usually start in seconds or less , Their images are usually megabytes, not gigabytes & One machine can run many of them --> Containers are light because they skip the extra OS. No boot, no full OS image.

Kubernetes - A Complete Guide in Tamil - 1

technonotes-hacker — Sun, 11 Oct 2026 07:26:24 +0000

helmsman
A helmsman is a person who physically steers a ship, boat, sailboat, submarine, airship, or spacecraft.

K8s
Most popular for managing the container.
Open source container orchestration platform.
orchestration --> Management of container.
Automate --> scale up & scale down , Deployment process.
Load Balacing

History

Developed by Google - 2014
CNCF --> Managing the k8s ( CNCF acquried ).
Kubernetes a greek word for helmsman.
7 spokes in k8s represents 7 founders.

Why we need k8s ?

To update all the container at a time with docker is a challange or complex process.
Container should not go down ( maintaince ).
If container fails , it needs to restart by itself.
Production Challenges : Manual scaling , HA & Networking along with storage.

K8s

All the above will be automated by KUBERNETES.
Automating deployment & scaling.
Self-healing.
LB
Managing storage.
Its like Swiggy Eg., Order FOOD and just relax . Swiggy will take care everything.

To host the application inside the containers.
We can't directly work with container , here its called PODS.
Pods will run on worker nodes.

Docker Swarm vs K8s

Both are event organiser.
Docker Swarm contains --> Cluster --> Cluster contains --> Nodes --> Nodes containes --> Containers --> Containers contain APPs.
K8s contains --> Cluster --> Cluster contains --> Nodes --> Nodes --> Nodes containes --> PODS --> PODS contain --> Containers contain APPs.

Docker Swarm --> Small scale events. Eg., Common wealth
K8s --> Large scale events. Eg., Olympics

Cluster

Cluster is a group of servers or virtual machine.
Any cluster contains

Master Node / Control Plane
Worker Nodes / Data Plane

Control Plane

Its like a director
Manage the scheduling , scaling & Monitoring.
Control Plane is like டைரக்டர் சங்கர். Control Plane எல்லாத்தையும் பாத்துப்பார்.

Worker Nodes / Data Plane

Its like actors
Follow the instruction of control plane and ensure the task is executed.
Each nodes will have containers inside.
The actual work will happen inside the worker node.
Each node runs containers inside PODS.
Worker Nodes is like ACTOR ரஜினிகாந்த் மாதிரி டைரக்டர் என்ன சொல்றாரோ அதை தான் கேட்கணும். Implementation இவர்தான்.

k8s Architecture

Each worker node containes:

Kubelet
Kube-Proxy

CLI --> Command Line Interphase
Kubectl --> With this only we are going to communicate with KUBERNETES.

How deployment Happens in K8s ?

You can't interact with WORKER NODE directly.
Anything you need to do in K8S means , you need to do with Control plane only.
You want to create a POD ? Inside the POD , you want to deploy a Application.
Now the user request will go to API Server and the API Server will store the request to etcd.
etcd --> Store the request in this and its a internal datasource of K8S.
Now Scheduler will always check the etcd , is there any request available.
Scheduler found one request that " someone trying to deploy the application ".
Scheduler will decide , in which WORKER NODE it needs to deployed.
How the Scheduler decides which worker node ? --> Scheduler will have a information of the WORKER NODE. ( storage , pods are running already , CPU ) will be assessed by the scheduler.
How Scheduler knows all the details of the worker node ? There will be worker node agent called --> KUBELET.
KUBELET role is to gather WORKER NODE information , so its called WORKER NODE AGENT. Which as all the informatio n related to the worker node and send the information to the Scheduler .
Scheduler will decide in which worker node it should be scheduled. thats it.
Who is the actuall one to deploy ? Controller-Manager.
Kube-Proxy is the network interphase between Control plane and worker node and between pods.
Kube-Proxy is a Network element of K8S Cluster --> Container Network Interphase --> CNI.
CNI is a plugin.
CNI will take care

Communication smoothly happens between Worker node and Control plane.
Control Plane and worker nodes.
Pods running in the same container.
Pods which are runninhg in same worker node.
Poda which are running different worker nodes.

Inside Woker Nodes , you will have CRI --> Container Runtime Interphase.
CRI --> is nothing but --> DOCKER.
CRI will help to create the pods and inside the pods , container will be availble.

All these will happen in FACTION OF SECS.

Types of K8S Cluster

Self Managed Cluster

Minikube --> Single node cluster
_Kubeadm _--> Multi Node Cluster --> Manual Maintainence.
Kops--> Multi Node Cluster --> Manual Maintainence.
KinD --> Multi Node Cluster --> Manual Maintainence. ( Kubernetes in Docker ) --> Kubernetes can also be maintained in Docker.
rancher --> Self Managed.

Cloud Managed K8S Cluster

AWS --> EKS --> Elastic Kubernetes Service
Azure --> AKS --> Azure Kubernetes Service
GCP --> Google Kubernetes Engine ( GKE )

Methods of creating Cluster

Direct Console
CLI --> Command Line Interphase
Manifest file --> Configuration file
Terraform

Billing of K8S

0.10$ per hour --> https://aws.amazon.com/eks/pricing/?trk=d04c06f6-db30-401a-b3fd-3b0ba3eaf643&sc_channel=ps&refid=d04c06f6-db30-401a-b3fd-3b0ba3eaf643

This is only for CLUSTER.
Other underlying resources will have separate charges.

AWS

EKS Cluster --> Elastic Kubernetes Service
Launch a EC2 Instance --> EKS Client VM
With this interact with the cluster.

Notes

Docker is used to create Container.
Docker Swarm is used to manage. Its a part of Docker.

Promotion - Canary Deployment

technonotes-hacker — Fri, 09 Oct 2026 03:45:31 +0000

a small, yellow songbird.
a small yellow bird that sings and is often kept as a pet.

What is Canary Deployment ?

Its a technique for rolling out new features or changes to a small traffic or instance before releasing the update to the entire system.
A small percentage of traffic is then routed to the new deployment , while the majority of the traffic continues to be served by the original deployment.
If the issues are detected during the canary deployment , it can be quickly rolled back to the original deployment.

Adavantage

Its a valuable tool for minimizing risk and ensuring high availability in complex distributed systems.

Target Deployment

Microservice

Process Flow

Plan and Create
Analyze
Roll Out

Implementation Plan

Deployment

Deployment team will take care via IR / CR accordyingly.

Canary Deployment will not be carried out for ?

DB changes
Low traffic Application
Request and Response changes
If the application as complex dependencies which one microservice will affect another in unexpected way then its difficult to isolate the issues.

How failures will be identified in canary deployment

POD wise logs will be maintained where the transaction failures can be verified.

Reference

Deploy a Release Using a Canary Deployment | Kubernetes

This tutorial walks you through deploying a canary release in Kubernetes. A canary deployment allows you to safely test a new application version with a small portion of production traffic before rolling it out completely. This approach helps minimize risk and enables rapid feedback from real users. For an overview of canary deployments, see Canary deployments in the Managing Workloads concept page. Objectives Deploy a stable version of an application Deploy a canary version alongside the stable version Route traffic to both versions using a Service Monitor the canary deployment Complete the rollout by scaling the canary and removing the stable version Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:

kubernetes.io

Promotion - UNIX / Linux Basics - 2

technonotes-hacker — Thu, 08 Oct 2026 18:05:54 +0000

The kernel is the core foundation of the OS that directly interfaces with system hardware and controls process and memory operations.
Kernal --> its between the computer's hardware and its processes.
SIGTERM (signal 15) is the default termination signal sent by commands like kill, allowing the process to perform clean-up routines.
graceful and clean termination , allowing the application to save its active transactions before closing --> SIGTERM (signal 15).
/etc/shadow contains the secure encrypted password hashes and security aging constraints. --> sensitive system file securely stores encrypted user password hashes and account expiration details , only root user can access.
30 2 * * 1-5 /opt/bank/scripts/eod_report.sh. Exactly when does this script run? --> 30 specifies the minute , 2 specifies the hour (2:30 AM), and 1-5 specifies weekdays (Monday through Friday).

top utility displays a live, refreshing summary of system resource usage and active tasks sorted by resource consumption. --> dynamic, continuous real-time view of active system tasks, CPU utilization metrics, and memory consumption profiles on a Linux server.
sar is used for collecting, reporting, and saving historical system activity data over time.
vmstat reports virtual memory, paging, and block IO statistics, but does not offer an interactive real-time process manager interface.
netstat inspects network connections, routing tables, and interface statistics.
inode --> File names are mapped to inode numbers inside directory files; the inode itself holds the metadata and block pointers, omitting the filename string. It stores file attributes, permissions, and disk pointers, but does NOT store the actual file name.

/proc/meminfo is a virtual kernel file that provides live metrics regarding total, free, cached, and swap memory usage. --> virtual file system path exposes detailed real-time kernel memory statistics and swap utilization data.

Notes

what is hardened enterprise Linux environment ?