RAID1 Ubuntu 16.04 P2V (physical to virtual conversion) using clonezilla Ubuntu 16.04

ispirace zde: https://www.youtube.com/watch?v=wSTk9BLwF5k&ab_channel=LawrenceSystems

video z instalace zde: https://maxbox.cz/nextcloud/index.php/s/jE2bzDi9TS7kHo5

Na stávajícím fyzickém serveru maxbox je Ubuntu 16.04.6 LTS (Xenial Xerus)

software RAID1 (2 x 2TB SATA HDD WDC WD2004FBYZ)

Struktura disků je následující

petr@maxbox:~$ sudo fdisk -l
Disk /dev/sda: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x4d84172a

Device     Boot      Start        End    Sectors  Size Id Type
/dev/sda1  *          2048  195311615  195309568 93.1G fd Linux raid autodetect
/dev/sda2        195313662 3907028991 3711715330  1.7T  5 Extended
/dev/sda5       3891404800 3907028991   15624192  7.5G fd Linux raid autodetect
/dev/sda6        195313664 3891404799 3696091136  1.7T fd Linux raid autodetect

Partition table entries are not in disk order.


Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x076a9f4c

Device     Boot      Start        End    Sectors  Size Id Type
/dev/sdb1  *          2048  195311615  195309568 93.1G fd Linux raid autodetect
/dev/sdb2        195313662 3907028991 3711715330  1.7T  5 Extended
/dev/sdb5       3891404800 3907028991   15624192  7.5G fd Linux raid autodetect
/dev/sdb6        195313664 3891404799 3696091136  1.7T fd Linux raid autodetect

Partition table entries are not in disk order.


Disk /dev/md0: 93.1 GiB, 99931389952 bytes, 195178496 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/md2: 7.5 GiB, 7995392000 bytes, 15616000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/md1: 1.7 TiB, 1892264443904 bytes, 3695828992 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Na ProxMoxu vytvoříme virtuální server linux (bez instalace konkrétního operačního systému) s velikostí disku 2 TB = 2048 GB = 2097152 MB 

Na tomto virtuálním serveru nabootujeme z CD Live Clonezilla

v Clonezilla odskok do terminálu

pomocí  sudo fdisk /dev/sda  

  • vytvořit disk DOS:   o   create a new empty DOS partition table
  • na něm vytvořit stejně velké oddíly (partitions) jako jsou na fyzickém (zdrojovém) serveru (Extended partition nevytvářet – není potřeba.)
  • nadefinovat typ oddílu na “Linx raid autodetect” (fd)
/dev/sda1  *          2048  195311615  195309568 93.1G fd Linux raid autodetect
/dev/sdb5       3891404800 3907028991   15624192  7.5G fd Linux raid autodetect
/dev/sdb6        195313664 3891404799 3696091136  1.7T fd Linux raid autodetect

Zdrojový server rebootovat a spustit na něm CD clonezilla live

  • vybrat klonování local partition to remote partition
  • vybrat první partition, kterou budeme přenášet.

Na cílovém stroji spustit CD Clonezilla live

  • vybrat part_to_remote_part
  • vybrat odpovídající target partition (partition se stejnou velikostí jako na zdrojovém sesrveru). 

Zahájit proces přenosu.

Takto opakovat pro všechny 3 partitions 

Na Proxmox reboot virtuálního serveru  (jako bootovací vybrat CD Ubuntu server disk).  Ve volbě na CD vybrat rescue mode.

Odskočit do shellu (Alt+F2)

cat/proc/mdstat
all disks inactive

z menu rescue Ubuntu vybrat

  • “assemble disc array”
  • partitons to assemble automatic 

 Alt+F2 vidime, ze jsou inactive

get detailed information about a RAID device 

 mdadm -D /dev/mdX  tvrdi RAID0 (ale je to nenačtené)

more detailed information by using the -E or –examine options:

mdadm  -E /dev/sda1,2,3 správně tvrdí RAID1 1 clean a 1 removed device, clean degraded

re-start RAID

mdadm -R /dev/md127
mdadm -R /dev/md126
mdadm -R /dev/md125

Run the following commands to change the array and filesystem back to read-write mode:

mdadm -w /dev/md127
mdadm -w /dev/md126
mdadm -w /dev/md125
dmesg
/dev/md125 switched to read-write mode

v menu Ubuntu “go back” + znovu “enter rescue mode”

rescue mode nabízí jako root uz i ty mdX

výběr toho, u ktereho  vedle v cat /proc/mdstat vidime, ze obsahuje sda1

v menu Ubuntu “reinstall GRUB boot loader”

zadat device /dev/sda

reboot from hdd (not from the Ubuntu rescue CD)

nabootuje Ubuntu 16.04

cat /proc/mdstat

ted uz jsou /dev/md0,1,2 

posledni uklid – neexistujici člen v RAID

mdadm --grow /dev/mdX --raid-devices=1 --force   … opet postupne

Test Kamery

					

Kitesurfing Rules of the Road

Original posters and the text files in English, Korean and Chinese are attached below.

You can help by translating them from English to other languages.

Kitesurfing Rules of the Road

  1. Before launching a kite from the beach, one must be sure that it does not create any danger, not only for other riders but also for other people on the launching spot. In addition, the fact that the kite can drag a few dozen meters during launching this should be taken into account. A rider launching his kite on the beach gives way to anybody around him.
  2. Give way to riders entering the water. Being on land with an inflatable kite is more hazardous than being on water. Always allow others to enter the water even if this means having to take another turn before being able to return to the beach.
  3. When passing one another, the upwind rider must put his kite up and downwind rider must put his kite down.
  4. Give way to riders who are not in control of their equipment. You should give space to riders without their board or who look less in control (students).
  5. If two riders are on colliding course, rider riding right hand forward has the priority. If rider is riding switch (toe side edge), he has left hand forward of course.
  6. When overtaking, faster kiter must not obstruct or impede the slower kiter. Faster kiter must keep clear by a suitable distance, either windward (upwind) or leeward (downwind).
  7. The downwind rider has the priority. The upwind rider keeps out of the way of the downwind rider.
  8. Rider riding a wave has always the priority.
  9. The multiperson meetings increase the risk of collisions and should be avoided. It concerns especially beginners. Seeing that in a moment the multiperson meeting will take place and having doubts whether it is safe, the best way is to make a U-turn on the opposite course. If there is no place for that, just stop, set your kite in the zenith and wait until the situation is cleared.
  10. Give way to other water users over. Remember, even if it is your right of way, it is your responsibility to avoid collision at all costs and to help keep kitesurfing safe and respect as a watersport.

Python MinimalModbus + SHT20 Modbus RTU RS485 Temperature and humidity sensor

#! /usr/bin/python
import minimalmodbus
import time
# Set the value minimalmodbus.CLOSE_PORT_AFTER_EACH_CALL=True 
# immediately after import minimalmodbus
minimalmodbus.CLOSE_PORT_AFTER_EACH_CALL=True

# default slave address = 1
instr = minimalmodbus.Instrument(port='/dev/ttyUSB0', slaveaddress=1)
instr.serial.baudrate = 9600
instr.serial.parity  = "N"
instr.serial.timeout  = 0.8
#instr.debug = True
print(instr)

"""
read slave address, baud rate, temp. correction, hum. correction
"""
print(instr.read_registers(257, 4,functioncode=3))

"""
change of the slave address 
the change will take effect after reboot only! 
it is possible to continue using the old address till reboot
"""
slaveNewAddress = 5
#instr.write_register(257, slaveNewAddress, functioncode=6)

"""
change of the speed - take effect imediatelly
the change will take effect after reboot only! 
it is possible to continue using the old address till reboot
"""
slaveNewBaudrate = 9600
#instr.write_register(258, slaveNewBaudrate, functioncode=6) 

# read slave address, baud rate, temp. correction, hum. correction
print(instr.read_registers(257, 4,functioncode=3)) 

time.sleep(1)

while 1:
    try:
        print instr.read_registers(1,2, functioncode=4) # read temp and hum (list)
        print instr.read_register(1,1, functioncode=4) # read temp 
        print instr.read_register(2,1, functioncode=4) # read hum 
    except minimalmodbus.NoResponseError:
        print("No Response Error")
    time.sleep(1)
root@danfoss:/home/pi/danfoss# sudo pip install -U minimalmodbus
Collecting minimalmodbus
  Downloading https://files.pythonhosted.org/packages/6c/d5/77d42e8a0b73da2b5f97acd91900ac50e303b4cb959f76350cfbb38e05a0/minimalmodbus-1.0.2-py2.py3-none-any.whl
Requirement already up-to-date: pyserial>=3.0 in /usr/local/lib/python2.7/dist-packages (from minimalmodbus)
Installing collected packages: minimalmodbus
  Found existing installation: MinimalModbus 0.7
    Uninstalling MinimalModbus-0.7:
      Successfully uninstalled MinimalModbus-0.7
Successfully installed minimalmodbus-1.0.2
root@danfoss:/home/pi/danfoss# 

Description of the module here

Power consumption (kWh) monitoring using node-red and influxDB

I have got nice 3phase 30(100)A four wire RS485 MODBUS Din Rail watt meter from Aliexpress (US $35.76) (max. baud rate is possible to set to 9600). I am able to read values using USB-RS485 adapter + Raspberry pi and sending data (json) to the mqtt server. So far no problem. Power meter is providing cummulated value (the same value you can read on the display).

At the first beginning I was sending the data in to the sqlite database using sqlite node-red node. The table was very simple:
(ID, TIMESTAMP, TOTAL_KWH).

The issue starts with the simple question: How to get daily (hourly, weekly, monthly… ) power consumption? Something like:
| 22.7.2019 | 450kWh|
| 23.7.2019 | 320kWh|
| 24.7.2019 | 300kWh|
At the first sight it looks as an easy task, but I ended up with this quite complex query. And as data was growing, the execution of the SELECT was slowing down.

The guys from the nice JOYSFERA server were pointing me to the influxDB. Installation was an easy part thanks to the:
https://www.osradar.com/install-influxdb-ubuntu-18-04-debian-9/

To import existing data in to the influxDB you need a text file with the following structure:

# DDL
CREATE DATABASE mrazirna

# DML
# CONTEXT-DATABASE: mrazirna
# CONTEXT-RETENTION-POLICY:autogen

power value=0.0 1564391161
power value=270.8 1564583065
power value=272.1 1564583462
power value=272.1 1564583542
power value=272.1 1564583621
power value=272.2 1564583701

While preparing the file, note the line ending has to be linux style and if you are importing big data-set, the error messages are displayed on the top of the import screen – you need to scroll up (which was kind of difficult to me to discover).
Now you are ready to import data in to the database:

> influx -import -path=/path/to/the/data/file -precision=s
2019/08/27 22:53:04 Processed 1 commands
2019/08/27 22:53:04 Processed 26934 inserts
2019/08/27 22:53:04 Failed 0 inserts

Next connect the node-red node to the influx database:
https://diyprojects.io/node-red-tutorial-saving-mysensors-measurements-on-influxdb/#.XWWvovxS9B8

When you are preparing data for the influxDB, remember you need to send the same data-type in to the database. In my case the value datatype was float

> show field keys
name: power
fieldKey fieldType
-------- ---------
value    float

It was necessary to writhe following (note *1.0 at the end of the first row) in to the node-red function node (otherwise the string was send to the influxDB)

msg.payload = msg.payload.POWER.toFixed(1)*1.0;
return msg;

The best part is the simplicity of extracting data:

> root@maxbox:/data/new# influx -precision rfc3339
Connected to http://localhost:8086 version 1.7.7
InfluxDB shell version: 1.7.7

> use mrazirna
Using database mrazirna

> select derivative(max(value)) from "power" where time > now()-7d group by time(1d) fill(0) tz('Europe/Prague')
name: power
time                      kWh
----                      ---
2019-08-21T00:00:00+02:00 200.19999999999982
2019-08-22T00:00:00+02:00 199.5
2019-08-23T00:00:00+02:00 202.10000000000036
2019-08-24T00:00:00+02:00 111.80000000000018
2019-08-25T00:00:00+02:00 309.89999999999964
2019-08-26T00:00:00+02:00 222.5
2019-08-27T00:00:00+02:00 278
2019-08-28T00:00:00+02:00 6.5

sqlite SELECT for power consumption reading

So far I have following working code (working with example data)

WITH miniPow as (
	select date(TIMESTAMP,'+1 day') as d, max(TOTAL_KWH) mini
	from power 
	group by date(timestamp)
	)
, maxiPow as (
	select date(TIMESTAMP) as d, max(TOTAL_KWH) maxi
	from power 
	group by date(timestamp)
	)
select maxiPow.d, ROUND(maxi-mini, 1) from miniPow 
 join 
maxiPow
on miniPow.d = maxiPow.d	

Below my first SQL attempt (not really elegant)

WITH consumption AS
  (SELECT date(TIMESTAMP) AS d,
          min(TOTAL_KWH) mi,
          max(TOTAL_KWH) ma
   FROM power
   GROUP BY date(TIMESTAMP)),
     enumerated AS
  (SELECT *,

     (SELECT count(*)
      FROM consumption b
      WHERE a.d >= b.d) AS cnt
   FROM consumption a),
     preenumerated AS
  (SELECT *,

     (SELECT count(*)-1
      FROM consumption b
      WHERE a.d >= b.d) AS cnt
   FROM consumption a)
SELECT strftime('%d.%m.%Y', en.d) AS "období od",
       strftime('%d.%m.%Y', pre.d) AS "období do",
       printf("%.1f", en.ma) AS "počáteční kWh",
       printf("%.1f", pre.ma) AS "koncový kWh",
       printf("%.1f", pre.ma-en.ma) AS "kWh za období",
       (JulianDay(pre.d) - JulianDay(en.d)) AS "počet dnů",
       printf("%.1f", (pre.ma-en.ma)/(JulianDay(pre.d) - JulianDay(en.d))) AS "kWh za den"
FROM enumerated en
JOIN preenumerated pre ON en.cnt = pre.cnt
UNION ALL
SELECT strftime('%d.%m.%Y', min(date(TIMESTAMP))) AS "období od",
       strftime('%d.%m.%Y', max(date(TIMESTAMP))) AS "období do",
       "" AS "počáteční kWh",
       "CELKEM:" AS "koncový kWh",
       printf("%.1f kWh", max(TOTAL_KWH)) AS "kWh za období",
       printf("za %.0f dnů", (JulianDay(max(date(TIMESTAMP))) - JulianDay(min(date(TIMESTAMP))))) AS "počet dnů",
       printf("ø %.1f kWh/den", (max(TOTAL_KWH)-min(TOTAL_KWH))/(JulianDay(max(date(TIMESTAMP))) - JulianDay(min(date(TIMESTAMP))))) AS "kWh za den"
FROM power
Link to example table here