mirror of
https://git.digitalstudium.com/digitalstudium/digitalstudium.com
synced 2023-12-29 08:06:35 +00:00
Initial commit
This commit is contained in:
Digital Studium
11
content/en/about-author.md
Normal file
11
content/en/about-author.md
Normal file
@@ -0,0 +1,11 @@
|
||||
---
|
||||
title: "About author"
|
||||
date: "2023-02-18"
|
||||
---
|
||||
|
||||
{{< figure src="/images/kostya.jpg" width="100px" class="floatleft alignleft" >}}
|
||||
My name is Konstantin Shutkin, I am from Moscow, Russia and I am a DevOps engineer. Since 2016, I have worked in this position in companies such as Severstal, Sberbank, Nvidia. My main subject of research and work is Linux, Docker, Kubernetes, Python programming and more. I share my knowledge on this blog.
|
||||
|
||||
My Github: https://github.com/digitalstudium
|
||||
|
||||
My email: konstantin@shootkin.ru
|
||||
107
content/en/linux-lifehacks/how-to-create-lvm-logical-volume.md
Normal file
107
content/en/linux-lifehacks/how-to-create-lvm-logical-volume.md
Normal file
@@ -0,0 +1,107 @@
|
||||
---
|
||||
title: "Linux: How to create LVM logical volume"
|
||||
date: "2022-05-15"
|
||||
---
|
||||
### First step: creating a physical volume
|
||||
After you have attached the disk to a physical server or virtual machine, you need to type
|
||||
this command:
|
||||
```bash
|
||||
sudo fdisk -l
|
||||
```
|
||||
<!--more-->
|
||||
to make sure the drive is recognized by the operating system, and to identify the drive name. Output
|
||||
of command will be something like this:
|
||||
```plaintext
|
||||
Disk /dev/vdb: 10 GiB, 10737418240 bytes, 20971520 sectors
|
||||
Units: sectors of 1 * 512 = 512 bytes
|
||||
Sector size (logical/physical): 512 bytes/512 bytes
|
||||
I/O size (minimum/optimal): 512 bytes/512 bytes
|
||||
```
|
||||
Once you have identified the drive name (in our case it is `/dev/vdb`), you can create
|
||||
physical volume using the command:
|
||||
```bash
|
||||
sudo pvcreate /dev/vdb
|
||||
```
|
||||
You will see output like this:
|
||||
```plaintext
|
||||
kostya@ubuntu-21-04:~$ sudo pvcreate /dev/vdb
|
||||
Physical volume "/dev/vdb" successfully created.
|
||||
kostya@ubuntu-21-04:~$
|
||||
```
|
||||
### Step two: create the volume group
|
||||
Now we need to create a volume group. This is done by the following command:
|
||||
```bash
|
||||
sudo vgcreate {vgname} {pvname}
|
||||
```
|
||||
In our case, the command will look like this:
|
||||
```bash
|
||||
sudo vgcreate vg-example /dev/vdb
|
||||
```
|
||||
The command output will look like this:
|
||||
```plaintext
|
||||
kostya@ubuntu-21-04:~$ sudo vgcreate vg-example/dev/vdb
|
||||
Volume group "vg-example" successfully created
|
||||
kostya@ubuntu-21-04:~$
|
||||
```
|
||||
### Step three: creating the logical volume
|
||||
Creating a logical volume can be done with the following command:
|
||||
```bash
|
||||
sudo lvcreate --size {size} --name {lv-name} {vg-name}
|
||||
```
|
||||
In our case, it will be:
|
||||
```bash
|
||||
sudo lvcreate --size 5G --name lv-example vg-example
|
||||
```
|
||||
You will see output like this:
|
||||
```plaintext
|
||||
kostya@ubuntu-21-04:~$ sudo lvcreate --size 5G --name lv-example vg-example
|
||||
Logical volume "lv-example" created.
|
||||
kostya@ubuntu-21-04:~$
|
||||
```
|
||||
If you want the logical volume to use all the free space in the volume group, then run the command:
|
||||
```bash
|
||||
sudo lvcreate --extents 100%FREE --name lv-example vg-example
|
||||
```
|
||||
### Fourth step: creating the filesystem
|
||||
To create an xfs filesystem, type the command:
|
||||
```bash
|
||||
sudo mkfs.xfs /dev/vg-example/lv-example
|
||||
```
|
||||
The command output will look like this:
|
||||
```plaintext
|
||||
kostya@ubuntu-21-04:~$ sudo mkfs.xfs/dev/vg-example/lv-example
|
||||
meta-data =/dev/vg-example/lv-example isize = 512 agcount = 4, agsize = 327680 blks
|
||||
= sectsz = 512 attr = 2, projid32bit = 1
|
||||
= crc = 1 finobt = 1, sparse = 1, rmapbt = 0
|
||||
= reflink = 1 bigtime = 0
|
||||
data = bsize = 4096 blocks = 1310720, imaxpct = 25
|
||||
= sunit = 0 swidth = 0 blks
|
||||
naming = version 2 bsize = 4096 ascii-ci = 0, ftype = 1
|
||||
log = internal log bsize = 4096 blocks = 2560, version = 2
|
||||
= sectsz = 512 sunit = 0 blks, lazy-count = 1
|
||||
realtime = none extsz = 4096 blocks = 0, rtextents = 0
|
||||
Discarding blocks ... Done.
|
||||
kostya@ubuntu-21-04:~$
|
||||
```
|
||||
To create an ext4 filesystem, replace the `mkfs.xfs` command with `mkfs.ext4`
|
||||
### Step Five: Mount the Logical Volume
|
||||
For example, suppose you want to mount the newly created logical volume to the `/opt` folder. In this
|
||||
case, add this line to file `/etc/fstab`:
|
||||
```bash
|
||||
/dev/vg-example/lv-example /opt xfs defaults 0 1
|
||||
```
|
||||
After that, type the command:
|
||||
```bash
|
||||
sudo mount -a
|
||||
```
|
||||
You can verify that the logical volume has been mounted successfully using the command:
|
||||
```bash
|
||||
df -h /opt
|
||||
```
|
||||
The output should be like this:
|
||||
```plaintext
|
||||
kostya@ubuntu-21-04:~$ df -h /opt/
|
||||
Filesystem Size Used Avail Use% Mounted on
|
||||
/dev/mapper/vg--random-lv--random 5.0G 68M 5.0G 2% /opt
|
||||
kostya@ubuntu-21-04:~$
|
||||
```
|
||||
@@ -0,0 +1,84 @@
|
||||
---
|
||||
title: "Linux: How to extend LVM volume"
|
||||
date: "2022-05-15"
|
||||
---
|
||||
### Situation 1: new disk
|
||||
#### First step: creating a physical volume
|
||||
After you have attached the disk to a physical server or virtual machine, you need to type command:
|
||||
```bash
|
||||
sudo fdisk -l
|
||||
```
|
||||
<!--more-->
|
||||
This is to make sure the drive is recognized by the operating system, and to identify the drive name. Output of the command will be something like this:
|
||||
```plaintext
|
||||
Disk /dev/vdc: 5 GiB, 5368709120 bytes, 10485760 sectors
|
||||
Units: sectors of 1 * 512 = 512 bytes
|
||||
Sector size (logical/physical): 512 bytes/512 bytes
|
||||
I/O size (minimum/optimal): 512 bytes/512 bytes
|
||||
```
|
||||
Once you have identified the drive name (in our case it is `/dev/vdc`), you can create physical volume using the command:
|
||||
```bash
|
||||
sudo pvcreate /dev/vdc
|
||||
```
|
||||
You will see output like this:
|
||||
```plaintext
|
||||
kostya@ubuntu-21-04:~$ sudo pvcreate /dev/vdc
|
||||
Physical volume "/dev/vdc" successfully created.
|
||||
kostya@ubuntu-21-04:~$
|
||||
```
|
||||
#### Step two: extend the volume group
|
||||
You can now extend the volume group. This is done by the following command:
|
||||
```bash
|
||||
sudo vgextend {vg-name} {pv-name}
|
||||
```
|
||||
In our case, it will be:
|
||||
```bash
|
||||
sudo vgextend vg-example /dev/vdc
|
||||
```
|
||||
You will see output like this:
|
||||
```plaintext
|
||||
kostya@ubuntu-21-04:~$ sudo vgextend vg-example /dev/vdc
|
||||
Physical volume "/dev/vdc" successfully created.
|
||||
Volume group "vg-example" successfully extended
|
||||
kostya@ubuntu-21-04:~$
|
||||
```
|
||||
#### Step three: extending the logical volume
|
||||
Extending a logical volume can be done with the following command:
|
||||
```bash
|
||||
sudo lvextend --size + {size} {vg-name/lv-name}
|
||||
```
|
||||
In our case, it will be:
|
||||
```bash
|
||||
sudo lvextend --size +2G vg-example/lv-example
|
||||
```
|
||||
You will see output like this:
|
||||
```plaintext
|
||||
kostya@ubuntu-21-04:~$ sudo lvextend --size +2G vg-example/lv-example
|
||||
Size of logical volume vg-example/lv-example changed from 5.00 GiB (1280 extents) to 7.00 GiB (1792 extents).
|
||||
Logical volume vg-example/lv-example successfully resized.
|
||||
kostya@ubuntu-21-04:~$
|
||||
```
|
||||
If you want the logical volume to use all the free space in the volume group, then type
|
||||
command:
|
||||
```bash
|
||||
sudo lvextend --extents +100%FREE vg-example/lv-example
|
||||
```
|
||||
#### Fourth step: extending the file system
|
||||
If you have xfs file system, then the extending can be done with the following command:
|
||||
```bash
|
||||
sudo xfs_growfs /dev/{vg-name}/{lv-name}
|
||||
```
|
||||
In our case, it will be:
|
||||
```bash
|
||||
sudo xfs_growfs /dev/vg-example/lv-example
|
||||
```
|
||||
For ext4 filesystem, replace `xfs_growfs` with `resize2fs`
|
||||
### Situation 2: if the size of the existing disk has changed
|
||||
Sometimes the size of an existing disk can change, for example, in case of a virtual machine. In this case, the first step will be different, the second step will not be performed, and the rest of the steps will be pretty the same as in the situation with a new disc described above. The first step is not to create a physical volume, but to resize the existing one. It can be done with the command:
|
||||
```bash
|
||||
sudo pvresize /dev/DISKNAME
|
||||
```
|
||||
For example,
|
||||
```bash
|
||||
sudo pvresize /dev/vdc
|
||||
```
|
||||
@@ -0,0 +1,60 @@
|
||||
---
|
||||
title: "Linux: how to limit /var/log folder size"
|
||||
date: "2022-06-16"
|
||||
---
|
||||
|
||||
Sometimes the `/var/log` folder grows so large that it causes a shortage of disk space. How to limit the size of this folder? By following the two steps in this article, you can control the size of the `/var/log` folder.<!--more-->
|
||||
### Step 1. Limiting the size of journald logs
|
||||
The logs of all systemd services are added to the `/var/log/journal/` folder by the `journald` service. To set the size limit for this folder, run the following commands:
|
||||
```bash
|
||||
sudo bash -c 'echo "SystemMaxUse=100M" >> /etc/systemd/journald.conf'
|
||||
sudo systemctl restart systemd-journald
|
||||
```
|
||||
Instead of the `100M` size, you can specify any other size, in `K, M, G, T` units. After the above commands, you can verify that the size of the folder `/var/log` has become the specified size using the command:
|
||||
```bash
|
||||
du -sh /var/log/journal/
|
||||
```
|
||||
### Step 2. Limiting the number of log files stored by logrotate
|
||||
Logrotate rotates almost all log files in the `/var/log` folder every day. For example, if I type the command `ls /var/log/kern*`, then I will see that in addition to the file `/var/log/kern.log` I have 4 more files stored that logrotate saved:
|
||||
```bash
|
||||
ls /var/log/kern*
|
||||
```
|
||||
```plaintext
|
||||
/var/log/kern.log /var/log/kern.log.2.gz /var/log/kern.log.4.gz
|
||||
/var/log/kern.log.1 /var/log/kern.log.3.gz
|
||||
```
|
||||
|
||||
To limit the number of log files, edit the file `/etc/logrotate.d/rsyslog`. Looking at the contents of the `/etc/logrotate.d/rsyslog` file, we can see that the default value of the `rotate` parameter is `4`:
|
||||
```bash
|
||||
cat /etc/logrotate.d/rsyslog
|
||||
```
|
||||
```plaintext
|
||||
/var/log/syslog
|
||||
/var/log/mail.info
|
||||
/var/log/mail.warn
|
||||
/var/log/mail.err
|
||||
/var/log/mail.log
|
||||
/var/log/daemon.log
|
||||
/var/log/kern.log
|
||||
/var/log/auth.log
|
||||
/var/log/user.log
|
||||
/var/log/lpr.log
|
||||
/var/log/cron.log
|
||||
/var/log/debug
|
||||
/var/log/messages
|
||||
{
|
||||
rotate 4
|
||||
weekly
|
||||
missingok
|
||||
notifempty
|
||||
compress
|
||||
delaycompress
|
||||
sharedscripts
|
||||
postrotate
|
||||
/usr/lib/rsyslog/rsyslog-rotate
|
||||
endscript
|
||||
}
|
||||
```
|
||||
You can change `4` to some other value, such as `1`, so that only one file is stored. In the `/etc/logrotate.d/` folder you will also find many other configuration files related to other log files, where you can also change the `rotate` setting.
|
||||
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
---
|
||||
title: "Linux: How to set up monitoring with alerts to Telegram"
|
||||
date: "2023-03-04"
|
||||
---
|
||||
This article describes how to set up monitoring with alerts to Telegram using Grafana, Prometheus, Alertmanager, Node-exporter and Cadvisor.
|
||||
### First step: Cloning the repository
|
||||
Log in to the server or go to local terminal and run the following commands:
|
||||
```bash
|
||||
git clone https://github.com/digitalstudium/grafana-docker-stack.git
|
||||
cd grafana-docker-stack
|
||||
git checkout alertmanager
|
||||
```
|
||||
<!--more-->
|
||||
### Second step: setting the external address of the server
|
||||
Open the `docker-compose.yml` file and on lines 22 and 38 change the address `127.0.0.1` to the address of the server where you want to install Prometheus.
|
||||
### Third step: creating a bot in Telegram
|
||||
In the Telegram search, type `Botfather` and follow the first link:
|
||||
|
||||
|
||||
|
||||
Then press `Start`:
|
||||
|
||||
|
||||
|
||||
After that, create a bot using the `Botfather` instructions.
|
||||
As a result, you should receive a message with an API token:
|
||||
|
||||
|
||||
|
||||
Copy this token and paste it on line 14 of the `configs/alertmanager.yml` file as the value of the `bot_token` parameter.
|
||||
|
||||
Then create a telegram group and add the created bot to this group. This group will receive notifications (alerts).
|
||||
|
||||
### Fourth step: Get the group id
|
||||
In the group you added the bot to, write some command, for example: `/my_id foobar`
|
||||
|
||||
Then in the browser follow the link
|
||||
`https://api.telegram.org/botINSERT_BOT_TOKEN_HERE/getUpdates` replacing `INSERT_BOT_TOKEN_HERE` with the token created in step 3.
|
||||
You should get something like this page:
|
||||
|
||||
|
||||
If there is nothing on your page, try sending the command `/my_id foobar` to the group again.
|
||||
|
||||
You need to copy the chat id value from this page and paste it on line 15 of the `configs/alertmanager.yml` file as the value of the `chat_id` parameter. Note that the value of the `chat_id` parameter must be without quotes and with a hyphen at the beginning.
|
||||
|
||||
### Fifth step: Deploying the docker stack
|
||||
While in the `grafana-docker-stack` folder, run the following commands:
|
||||
```bash
|
||||
docker swarm init
|
||||
docker stack deploy -c docker-compose.yml monitoring
|
||||
```
|
||||
Wait 5-10 minutes then run the command
|
||||
```bash
|
||||
docker ps
|
||||
```
|
||||
|
||||
The command output should contain 5 containers: prometheus, grafana, alertmanager, node-exporter, grafana.
|
||||
```plaintext
|
||||
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
|
||||
46fba26e7234 gcr.io/cadvisor/cadvisor:v0.47.0 "/usr/bin/cadvisor -…" 5 days ago Up 5 days (healthy) 8080/tcp monitoring_cadvisor.1.q02qcn798dh0rydo1dzslylse
|
||||
f212e3c66786 prom/alertmanager:v0.25.0 "/bin/alertmanager -…" 6 days ago Up 6 days 9093/tcp monitoring_alertmanager.1.oysziztrqnur7xr0hr82avunz
|
||||
c16fb14929e2 prom/prometheus:v2.42.0 "/bin/prometheus --c…" 6 days ago Up 6 days 9090/tcp monitoring_prometheus.1.yslspi4fgjp7ic4f5e18gm99a
|
||||
9bf01ce6b7a1 grafana/grafana:9.3.6-ubuntu "/run.sh" 6 days ago Up 6 days 3000/tcp monitoring_grafana.1.mypn85x12xw37ucprr33knuwk
|
||||
58efdb46f5c3 kindest/node:v1.25.3 "/usr/local/bin/entr…" 6 days ago Up 6 days 127.0.0.1:46579->6443/tcp kind-control-plane
|
||||
ae15e453e517 prom/node-exporter:v1.5.0 "/bin/node_exporter …" 7 days ago Up 7 days 9100/tcp monitoring_node-exporter.1.uecim10ow12h1qlpox5lg0c5r
|
||||
```
|
||||
If you see this output, then everything turned out successfully. If not, try repeating all previous steps.
|
||||
|
||||
### Sixth step: Check if it works
|
||||
|
||||
Go to [server ip-address or domain name]:3000
|
||||
You should get Grafana opened. Enter login `admin` and password `admin`. Grafana will ask you to change your password, change it to any other.
|
||||
Under Dashboard -> Browse -> General you will see 2 dashboards: Cadvisor exporter and Node Exporter Full. Open them and make sure everything works.
|
||||
|
||||
## Summary
|
||||
Now you can evaluate server performance through Grafana graphs.
|
||||
You will also receive alerts about problems with the server in the Telegram group. Alert rules can be configured in `configs/prometheus/alert_rules.yml`
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: "Python: How to load multiple web pages in parallel"
|
||||
date: "2022-05-15"
|
||||
---
|
||||
First you need to install an aiohttp package. To install aiohttp run the command:
|
||||
```bash
|
||||
pip install aiohttp[speedups]
|
||||
```<!--more-->
|
||||
The `[speedups]` suffix is needed to install aiohttp accelerating packages - aiodns and cchardet. Then create a main.py file with this code:
|
||||
```python
|
||||
import aiohttp
|
||||
import asyncio
|
||||
import socket
|
||||
|
||||
|
||||
async def fetch_urls(urls):
|
||||
resolver = aiohttp.AsyncResolver()
|
||||
connector = aiohttp.TCPConnector(resolver=resolver, family=socket.AF_INET, use_dns_cache=False)
|
||||
session = aiohttp.ClientSession(connector=connector)
|
||||
|
||||
async def fetch_url(url, session):
|
||||
async with session.get(url) as resp:
|
||||
print(resp.status)
|
||||
print(await resp.text())
|
||||
|
||||
tasks = [fetch_url(url, session) for url in urls]
|
||||
await asyncio.gather(*tasks)
|
||||
await session.close()
|
||||
|
||||
|
||||
loop = asyncio.get_event_loop()
|
||||
|
||||
urls = ['http://httpbin.org/get?key=value1', 'http://httpbin.org/get?key=value2', 'http://httpbin.org/get?key=value3']
|
||||
|
||||
loop.run_until_complete(fetch_urls(urls))
|
||||
|
||||
```
|
||||
Now you can run main.py file with the command:
|
||||
```bash
|
||||
python3 main.py
|
||||
```
|
||||
You will see this output:
|
||||
```plaintext
|
||||
200
|
||||
{
|
||||
"args": {
|
||||
"key": "value2"
|
||||
},
|
||||
"headers": {
|
||||
"Accept": "*/*",
|
||||
"Accept-Encoding": "gzip, deflate",
|
||||
...
|
||||
|
||||
```
|
||||
All three queries will be executed in parallel. You can add any urls to the `urls` list, for example:
|
||||
```python
|
||||
urls = ['https://yandex.com', 'https://google.com', 'https://yahoo.com']
|
||||
```
|
||||
|
||||
In order to make HEAD, POST, PUT, DELETE requests, just replace `session.get(url)` in your code with the appropriate method:
|
||||
```python
|
||||
session.post('http://httpbin.org/post', data=b'data')
|
||||
session.put('http://httpbin.org/put', data=b'data')
|
||||
session.delete('http://httpbin.org/delete')
|
||||
session.head('http://httpbin.org/get')
|
||||
session.options('http://httpbin.org/get')
|
||||
session.patch('http://httpbin.org/patch', data=b'data')
|
||||
|
||||
```
|
||||
23
content/en/ubuntu-lifehacks/ubuntu-how-to-upgrade-kernel.md
Normal file
23
content/en/ubuntu-lifehacks/ubuntu-how-to-upgrade-kernel.md
Normal file
@@ -0,0 +1,23 @@
|
||||
---
|
||||
title: "Ubuntu: How to upgrade kernel"
|
||||
date: "2022-05-14"
|
||||
---
|
||||
### First method
|
||||
The first method is very simple. We need to enter only one command in the terminal:
|
||||
```bash
|
||||
sudo apt update && sudo apt -y upgrade
|
||||
```<!--more-->
|
||||
The `sudo apt update` command will update the repository cache, and the `sudo apt -y upgrade` command will install new versions of all installed programs, including the linux kernel. The advantage of this method is that the latest version of the linux kernel, <i>officially supported</i> by Ubuntu OS, will be installed. The disadvantage of this method is that the <i>officially supported</i> kernel is usually not the newest. Sometimes it happens that it is necessary to install the latest version of the linux kernel. Real world example: your new laptop may have a CPU which is only supported in linux kernel version 5.12, while the officially supported version is older. And here the second method comes to the rescue.
|
||||
### Second method
|
||||
The first step is to go to https://kernel.ubuntu.com/~kernel-ppa/mainline/. On this site, you need to select the folder with the latest version of the linux kernel (at the very bottom of the page). Note that it is recommended to select the version without the "rc" suffix. The "rc" suffix means "release candidate", which in turn means that the given kernel version is not stable. On the page that opens, select the folder with the architecture of your processor. The architecture can be found using the `uname -p` command. If the output of this command is "x86_64", then select the amd64 folder. On the opened page there will be links to .deb files. We need to download 4 of them:
|
||||
```plaintext
|
||||
linux-headers-{version}-generic_{version}.{date}_amd64.deb
|
||||
linux-headers-{version}_{version}.{date}_all.deb
|
||||
linux-image-unsigned-{version}-generic_{version}.{date}_amd64.deb
|
||||
linux-modules-{version}-generic_{version}.{date}_amd64.deb
|
||||
```
|
||||
After you have downloaded the files, you need to install them using the command:
|
||||
```bash
|
||||
sudo apt install -y ~/Downloads/linux-*.deb
|
||||
```
|
||||
Installation will take 1-5 minutes. After restarting the computer, the installed kernel version will be loaded.
|
||||
Reference in New Issue
Block a user