Python Virtual Environment

 python -m venv my_env

source my_env\bin\activate

pip install jupyter

jupyter notebook

Google Collaborator:

!pip install -q langchain

Secrets:

from google.colab import userdata

api_key = userdata.get('API_KEY')

Gemini: ai.google.dev/available_regions

VPN: protonvpn

Installing google gen ai: pip install -q google-generativeai

pip install --upgrade -q google-generativeai

pip show google-generativeai

pip install python-dotenv

import google.generativeai as genie

import os

from dotenv import load_dotenv, find_dotenv

load_dotenv(find_dotenv(), override=True)

# os.environ.get("GOOGLE_API_KEY")

genie.configure(api_key=os.environ.get("GOOGLE_API_KEY"))

for m in genie.list_models():

    if 'generateContent' in m.supported_generation_methods:

        print(m.name)

Accessing Google AI Studio: ai.google.dev

Docker

 Docker virtualizes Application Layer of an Operating System.

It runs on top of the native Operating System Kernel layer. In this way, Docker images are entirely running in the application space.

Commands

1. Find Docker images on the local machine: docker images
2. Find Docker containers running on the local machine: docker ps
3. Download image version from DockerHub: docker pull nginx:1.23
4. Download latest image from DockerHub: docker pull nginx
5. Run image of specific version: docker run nginx:1.23
6. Run container in the background: docker run -d nginx:1.23
7. Print container logs to terminal: docker logs <containerid>
8. 
   

Mac Command Line and Tools

1. Check ports in use
    sudo lsof -i -n -P 

2. Check if postgres processes are running
    ps -ef | grep postgress

Debug processes with high memory usage

When a process erroneously holds on to excessive memory and not release it, it could eventually lead to under performance or crash. If the memory is being held in heap, try the following:

1. Get a process dump
2. Analyze the process in Windbg by running the following commands

!address -summary
!heap -s
!heap -stat -h <heap address>
!heap -flt s <size>
!heap -p -a <memory address>

!address -summary
0:000> !address -summary
Building memory map: 00000000Building memory map: 00010000
Mapping file section regions...
Mapping module regions...
Mapping PEB regions...
Mapping TEB and stack regions...
Mapping heap regions...
Mapping page heap regions...
Mapping other regions...
Mapping stack trace database regions...
Mapping activation context regions...

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Heap                                    790          5adf9000 (   1.420 GB)  74.95%   71.00%
Stack                                   840          11800000 ( 280.000 MB)  14.43%   13.67%
<unknown>                              1325           81f6000 ( 129.961 MB)   6.70%    6.35%
Free                                    949           6c04000 ( 108.016 MB)            5.27%
Image                                   767           4a7d000 (  74.488 MB)   3.84%    3.64%
TEB                                     280            118000 (   1.094 MB)   0.06%    0.05%
Other                                    58             67000 ( 412.000 kB)   0.02%    0.02%
PEB                                       1              1000 (   4.000 kB)   0.00%    0.00%

--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_PRIVATE                            3188          73d7b000 (   1.810 GB)  95.54%   90.50%
MEM_IMAGE                               789           4cc0000 (  76.750 MB)   3.96%    3.75%
MEM_MAPPED                               84            9b1000 (   9.691 MB)   0.50%    0.47%

--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_COMMIT                             3168          5fa20000 (   1.494 GB)  78.88%   74.72%
MEM_RESERVE                             893          199cc000 ( 409.797 MB)  21.12%   20.01%
MEM_FREE                                949           6c04000 ( 108.016 MB)            5.27%

--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal
PAGE_READWRITE                         1878          5a623000 (   1.412 GB)  74.55%   70.61%
PAGE_EXECUTE_READ                       232           2b4a000 (  43.289 MB)   2.23%    2.11%
PAGE_READONLY                           436           2289000 (  34.535 MB)   1.78%    1.69%
PAGE_READWRITE|PAGE_GUARD               560            47b000 (   4.480 MB)   0.23%    0.22%
PAGE_WRITECOPY                           61            1a4000 (   1.641 MB)   0.08%    0.08%
PAGE_EXECUTE_READWRITE                    1              b000 (  44.000 kB)   0.00%    0.00%

--- Largest Region by Usage ----------- Base Address -------- Region Size ----------
Heap                                        1a6b0000            fd0000 (  15.813 MB)
Stack                                        1a60000             fd000 (1012.000 kB)
<unknown>                                   7f0e0000            f00000 (  15.000 MB)
Free                                         ba0f000             f1000 ( 964.000 kB)
Image                                       767b3000            879000 (   8.473 MB)
TEB                                         7e898000              1000 (   4.000 kB)
Other                                       7efb0000             23000 ( 140.000 kB)
PEB                                         7efde000              1000 (   4.000 kB)

!heap -s

************************************************************************************************************************

                                              NT HEAP STATS BELOW

************************************************************************************************************************

LFH Key                   : 0x37db7eb7

Termination on corruption : ENABLED

  Heap     Flags   Reserv  Commit  Virt   Free  List   UCR  Virt  Lock  Fast 

                    (k)     (k)    (k)     (k) length      blocks cont. heap 

-----------------------------------------------------------------------------

Virtual block: 43a90000 - 43a90000 (size 00000000)

00770000 00000002 1411920 1402508 1411920  12100  3140   421    1    56a   LFH

00130000 00001002    1088    888   1088    170    18     2    0      0   LFH

00580000 00001002   57680  56328  57680   1723   890    38    0    202   LFH

00670000 00001002     256    256    256      9     9     1    0      0   LFH

...

-----------------------------------------------------------------------------

Output shows that virtual block 00770000 is holding 1411920 k memory.

!heap -stat -h 00770000

heap @ 00770000

group-by: TOTSIZE max-display: 20

    size     #blocks     total     ( %) (percent of total busy bytes)

    78 5e6c95 - 2c42e5d8  (57.94)

    104 8647c - 8860df0  (11.16)

    fc 80000 - 7e00000  (10.31)

    20 2a10c2 - 5421840  (6.88)

    ...

Output shows that 57.94% of blocks are of size 78.

!heap -flt s 78
shows a number of pointers that hold onto blocks of size 78

!heap -p -a <memory address>

shows the stack at memory address 

Reference: https://www.codeproject.com/Articles/31382/Memory-Leak-Detection-Using-Windbg

git tutorial

Sample git operations

1. git clone <full path of git branch>
2. git fetch && git checkout <name of branch>
3. git pull
4. git status
5. git add .
6. git status
7. git commit -m "comment"
8. git push origin <name of branch>

Explanation:

1. git clone <full path of git branch>

   Clones the repository from remote server to the local computer

2. git fetch && git checkout <name of branch>

   Prepares local repository for updates

3. git pull

    Downloads any changes that were performed on the remote repository to local repository

4. git status

    Shows which files in the local repository have been updated along with the files that were registered and not registered for updates

5. git add .

    Registers all files that were modified in the local repository 

    You may also specify individual files by executing git add <filepath>

6. git status - look at item 4

7. git commit -m "comment"

    Commits all the files that were added using git add to the local repository. This is also called as Staging

8. git push origin <name of branch>

    Updates the remote repository with the changes that were committed in local repository

Common commands in git:

1. Show me what files are changed

git status

2. Stage the files (-A stages all modified files)

    git add -A

3. To get to the latest committed files

    git checkout -- .

4. Version of git

    git --version

5. Configure git with username and email

    git config --global user.name "username"

    git config --global user.email "youremail@address"

6. Compare updates done to a file in local repository

   git diff <name of file>

Ionic Tutorial

Summary:

Create a new Ionic Application: C:\Projects> ionic start myapp APPNAME
CD into the project:                     C:\Projects> cd APPNAME
Setup project to use Sass:             C:\Projects\APPNAME> ionic setup sass
Develop in the browser with live reload:  C:\Projects\APPNAME> ionic serve
Add platform (iOS - OS X or Android):     C:\Projects\APPNAME> ionic platform add ios [android]
Build App:                                 C:\Projects\APPNAME> ionic build <PLATFORM>
Simulate App:                             C:\Projects\APPNAME> ionic emulate <PLATFORM>
Run App:                                   C:\Projects\APPNAME> ionic run <PLATFORM>
Package App:                              C:\Projects\APPNAME> ionic package <MODE> <PLATFORM>
Invite to preview                       C:\Projects\APPNAME> ionic share EMAIL


Installation issues:
1. While installing on MacBook, if you get an EACCESS error while installing to /usr/local, do the following
    sudo chown -R $USER /usr/local

References:
1. Youtube Video: https://www.youtube.com/watch?v=C-UwOWB9Io4&feature=youtu.be
2. Generate splash screens and icons with ionic resource
    http://ionicframework.com/blog/automating-icons-and-splash-screens/
3, Ionic View App
    http://view.ionic.io
4. Tutorial - scotch-todo simple
    https://scotch.io/tutorials/create-your-first-mobile-app-with-angularjs-and-ionic
5. Tutorial - nraboy todo with SQLite:
    https://blog.nraboy.com/2015/03/create-todo-list-mobile-app-using-ionic-framework/
6. Data storage on Android:
    http://developer.android.com/guide/topics/data/data-storage.html

Memory management of BSTR objects in COM

When working with BSTR with COM objects, the following questions have to be answered:
1. When passing BSTR to a COM object, who is responsible to free the memory allocated to BSTR?
2. When a COM object returns BSTR, who is responsible to free the memory allocated to BSTR?

Answer to both these questions is: caller of the COM object is responsible for freeing memory allocated to BSTR.

Examples:
1. When passing BSTR to COM object
Solution:
    {
            CComPtr<IComDog> dog;
            dog.CoCreateInstance(Animals::CLSID_Dog);
            BSTR nameOfDog = SysAllocString(L"Tiger");
            dog->put_NameOfDog(nameOfDog);
            .....
            // other code for dogs
            .....
            SysFreeString(nameOfDog);
            nameOfDog = NULL;
    }

Instead of using BSTR directly, use CComBSTR which takes care of memory management.
    {
            CComPtr<IComDog> dog;
            dog.CoCreateInstance(Animals::CLSID_Dog);
            CComBSTR nameOfDog = L"Tiger";
            dog->put_Name(nameOfDog);
            .....
            // other code for dogs
            // free'ing memory allocated to nameOfDog is taken care of by CComPtr
     }

2. When COM return a BSTR
     TBD

PowerShell Script to find Process IDs of an Executable

To find the priority of Threads of a process, copy the following script and execute in PowerShell

$timeoutInSeconds = 100;
$processName = "myExecutable.exe"

$processList = Get-WmiObject win32_process | where { $_.ProcessName -eq "$processName" } | select Handle
foreach ($processIds in $processList)
{
    $processId = $processIds.Handle
    Write-Host "process id = $processId"
}


Disclaimer: Use at your own risk. The author does not take any responsibility for the behavior of this script.

Lock Windows Desktop during Remote Session

In order to lock a Windows desktop during remote session, execute the following command

rundll32.exe user32.dll,LockWorkStation

Common WinDbg commands

How to set Symbol path
srv*c:\symbols*http://msdl.microsoft.com/download/symbols;
srv*c:\symbols*c:\temp\localsymbols;
srv*c:\symbols*\\network_share\symbols;

or
set environment variable name: _NT_SYMBOL_PATH
set environment variable value: srv*c:\symbols*http://msdl.microsoft.com/download/symbols;
                     srv*c:\symbols*c:\temp\localsymbols; srv*c:\symbols*\\network_share\symbols;

How to load Wow64 on x64 debugger
.load wow64exts

Other commands
lml
.ecxr
!analyze -v

How to verify if the module and pdb are a match
!itoldyouso <ModuleName> <ModulePDB>

Setup WinDbg as Postmortem Debugger
windbg -I

Setup ProcMon as Postmortem Debugger
procmon -i -ma

Redirect output to a log file
000> .logopen c:\temp\hello.txt
000> <do what ever commands> - all output is shown in windbg window and redirected to log file
000> .logclose

Find machine name (.NET dmp)
kd> x srv!SrvComputerName
fffff880`06ec4540 srv!SrvComputerName = <no type information>
kd> dq fffff880`06ec4540
fffff880`06ec4540  00000000`001a001a fffff8a0`02441df0
fffff880`06ec4550  00000000`00000000 00000000`00000000
fffff880`06ec4560  00000000`00000000 00000000`000c000a
...
kd> du fffff8a0`02441df0
fffff8a0`02441df0  "ABC"

Read GUID from a PDB using WinDbg

Here are the steps for reading GUID of a PDB

1. Install WinDbg
2. Set symbol path to the location of PDB
3. Load any executable into WinDbg
4. Execute in WinDbg 'ld <ModuleName>'
5. Execute in WinDbg '!chksym <ModuleName>
    c:\temp\<name_of_module>.dll
    Timestamp: 53FEA125
    SizeOfImage: DE000
    pdb: c:\temp\sample.pdb
    pdb sig: aaaaaaaa-bbbb-bbbb-bbbb-aaaaaaaaaaaa
    age: 1

Kill Remote Process using PowerShell

Here are the steps to kill a process running on remote machine

1. Start PowerShell
2. $processes = Get-WmiObject -class win32_process -ComputerName <name_of_computer> -Filter "Name='notepad.exe'"
3. $x = $processes.terminate()
4. echo $x

Return value should be 0

const char * vs char * const

I came across the question 'What is the difference between const char * vs char * const many times. After trying several sample programs, here is a simple way to understand the differences. Let us see what char * means:

char * p; // defines a buffer where characters can be stored
          // returns a pointer that is pointing to the first element in the buffer and is denoted by p

When const is applied on char *, depending on the position of const,
different things happen on the buffer+pointer.
const char * p ->  buffer contents are a constant. Pointer variable p can be modified to point to a different buffer.
char * const p ->  buffer contents can be modified. Pointer variable p cannot be modified. p points to the same location, all the time.
const char * const p -> means that the buffer and the pointer cannot be modified

Summary:

Definition
Buffer
Pointer
const char * p
Cannot change
Can change
char * const p
Can change
Cannot change
const char * const p
Cannot change
Cannot change

Linux Commands

A list of Linux commands

How To
Commands
Disable Firewall
$ service iptables stop   # stops firewall
$ chkconfig iptables off  # disables firewall
$ service iptables status # status of firewall 

Renew IP Address

When computers are moved from one network to another network, its IP address changes. This change is not reflected immediately in network adapters. In order to force a reset of these adapters, the following commands are used.

Renew IP Address

Windows
1. Open a cmd prompt in elevated mode
2. Execute the command
    c:\windows\system32> ipconfig /renew


Linux
1. Open a Terminal as root
2. Execute the command
    $ ifconfig eth0 down
    $ ifconfig eth0 up

Schema conversion - AUTO_INCREMENT (MySQL) and SEQUENCE (PostgreSQL)

Auto increment is achieved in MySQL using the keyword AUTO_INCREMENT. It is achieved in PostgreSQL using a SEQUENCE statement. Consider the following problem: Create a table with a field, to auto increment starting with the value 5.

MySQL
CREATE TABLE hello (
    idHello int NOT NULL AUTO_INCREMENT
) AUTO_INCREMENT=5;


PostgreSQL
CREATE SEQUENCE hello_idHello_sequence START 5;
CREATE TABLE hello (
    idHello integer NOT NULL DEFAULT nextval('hello_idHello_sequence')
);
ALTER SEQUENCE hello_idHello_sequence OWNED BY hello.idHello;

Threat Modeling - STRIDE

While performing threat modeling on software, verify that every data item, end point crossing a boundary, open port etc are subject to the categorization defined by STRIDE. Each item under consideration may fall under more than one category.

STRIDE Definition

Spoofing User: Accessing and using a resource by masquerading as another user 

Tampering Data: Modifying data maliciously, so that the original data is unavailable and/or malicious                              data is introduced

Repudiation: Denying by a user or system, that an action has been performed by them

Information Disclosure: Disclosure of information to an unauthorized user or system

Denial of Service: Denying the Service to legitimate users or system

Elevation of Privilege: Unauthorized elevation of privileges by an unprivileged user

For more information, refer to Security Design by Threat Modeling

Tomcat - Application System Error - Access is Denied

Application: Tomcat
OS: Windows
Error: Unable to open the service 'Tomcat7'

When you install Tomcat on a Windows machine, the following error is seen on rebooting the machine.

Reason: Tomcat7w.exe accesses resources that require administrator privileges.

Solutions: Configure Tomcat7w.exe to run as administrator. To configure Tomcat7w.exe run as administrator

• Right-click on Tomcat7w.exe
• Click on Properties
• Click on Compatibility
• Under Privilege Level, check the box 'Run this program as an administrator'
• Click OK
• Reboot the machine

Programs for Windows Administration

This is a list of programs, that you can use for Windows administration.

1. Computer Management: compmgmt.msc
2. Component Services: comexp.msc 
3. Event Viewer: eventvwr.exe 
4. Local Security Policy: secpol.msc 
5. ODBC Data Sources: odbcad32.exe
6. Performance Monitor: perfmon.msc
7. Print Management: printmanagement.msc
8. Services: services.msc
9. System Configuration: msconfig.exe 
10. Task Scheduler:taskschd.msc
11. Windows Firewall with Advanced Security: WF.msc 

Linux Commands

Here are some common Unix/Linux commands

1. OS bitness 32 or 64: getconf KERNEL_BITS
2. architecture of processor: file /usr/bin/w
3. disk space usage: du
4. enable backspace: stty erase hit_the_backspace_key
5. split a vim screen vertically: Ctrl+wv
6. list of open ports: netstat
7. list of processes: ps
8. rename a file: mv original_filename renamed_filename
9. move a file: mv filepath/filename target_filepath/filename
10. change permissions: chmod
11. change owner: chown