summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* live: rewrite software inventoryv1.2.7Jose M. Guisado2023-03-272-19/+134
| | | | | | | | | | | | | | | | | | | | | | | | | | | Replace legacy bash script in favor of Python code. Improves error traceability and further development. The software inventory operation mounts the target partition and it fetches the list of installed software (package set). Once the operation is complete, it unmounts the target partition. For Windows, introduce hivex library python bindings for accessing Windows registry hive files (https://libguestfs.org/hivex.3.html). This operation is still processed by legacy code in the server side (ogAdmServer.c in ogServer). Legacy backend process expects the software inventory like the following example: "software": "Windows 10 Enterprise Evaluation 2004 \nIntel(R) Network Connections 24.0.0.11 24.0.0.11 ..." The os name is inserted first in this list followed by a '\n' separated string of the software packages. The legacy server code can be found in function actualizaSoftware at ogServer/src/ogAdmServer.c It is expected for software inventory payload to change in the future to a simpler solution using just a json array of strings.
* live: fix ogChangeRepo invocationJose M. Guisado2023-03-271-1/+1
| | | | | | | | | | | | | Add missing samba credentials parameter in ogChangeRepo invocation. Credentials are loaded from ogClient config file. Any production deployment should use its own samba user and password. ogChangeRepo fails when using default samba credentials in a production environment. Fixes: a1edbe904b30 ("legacy: rewrite ogChangeRepo") Fixes: 3703fd60634f ("live: support native unicast cache image restore")
* live: fix unicast cache error log messageJose M. Guisado2023-03-271-1/+1
| | | | | | | | | Removes undefined 'repo' variable from error logging message. This caused the traceback to be polluted with an unhelpful message about this variable being undefined. Fixes: 3703fd606 ("live: support native unicast cache image restore")
* live: add linux swap parttypeJose M. Guisado2023-03-231-10/+12
| | | | | | | Adds linux swap partition type, mapped to the 'LINUX-SWAP' string in web interfaces like ogCP or webconsole. Fixes: 29c53e54e945 ("live: add parttypes.py")
* ogRest: improve error logging when executing operationsv1.2.6Jose M. Guisado2023-03-101-32/+23
| | | | | | | | | | | | | Capture all possible Python exceptions in the try/except block of every opengnsys operation. Create an error handling function to deduplicate code in the except block. The error handling function resets the ogRest state to IDLE and sends the corresponding 500 Internal Server Error. This *does not cover* every possible error. There are functions inside ogThread which contain code that may raise errors that are not covered by any try/except block.
* ogRest: remove root logger constantJose M. Guisado2023-03-101-7/+6
| | | | | | | | | | | | | | | | Remove unnecesary root logger constant: LOGGER The root logger is used by default when executing: logging.debug() logging.info() logging.warning() ... There is no point in doing: LOGGER = logging.getLogger() # Get root logger LOGGER.debug() # Use root logger
* legacy: improve readability of ogGetImageInfo helper functionsJose M. Guisado2023-03-101-14/+13
| | | | | | | | | | | | | | | | | | | | Change the name of the helper functions used when getting opengnsys image information (legacy ogGetImageInfo bash script). As of now the process consist of decompressing the image file with lzop and feeding that output to partclone.info. Prefer a more explicit function name rather than "process_image_*" Add comment about skipping the first two lines of partclone.info output. Usually, partclone.info starts printing out these two lines that are not related to the partclone image information: Partclone v0.3.23 http://partclone.org Showing info of image (-) As long as partclone.info output doesn't change we'll be fine, but we should not depend on human readable output. This might change in the future (i.e. adding json output format to partclone.info).
* legacy: rewrite ogGetImageInfoJose M. Guisado2023-03-022-18/+108
| | | | | | | | | | | | | | | | | | | | Rewrites this legacy script behavior using native Python code, using subprocess module when executing programs like partclone.info or lzop ogGetImageInfo is a bash script that retrieves information regarding an OpenGnsys partition image, specifically: - clonator - compressor - filesystem - datasize (size of the partition image) This rewrite only supports partclone and lzop compressed images. This is standard behavior, we have no reports of other programs or compression algorithms in use. Keep this legacy function with hungarian notation to emphasize this is still a legacy component that may be replaced in the future.
* legacy: rewrite ogChangeRepoJose M. Guisado2023-03-022-7/+53
| | | | | | | | | | | | | | | | Drop ogChangeRepo Bash script in favor of a native Python approach. Use only necessary subprocess calls instead of bringing all the logic of this function into a Bash script black box. ogChangeRepo unmounts the current OpenGnsys image samba folder (/opt/opengnsys/images) and mounts (connects to) a new directory using the new provided ip address. Keeping access mode from previous mount. If anything goes wrong when mounting the new directory, it will fallback to mounting the previous directory. If no previous OpenGnsys image samba directory is detected, this functions tries to mount the new directory anyway. In this case, it will raise CalledProcessError if something goes wrong.
* legacy: improve readability of cambiar_accessoJose M. Guisado2023-03-021-10/+10
| | | | | Expand function docstring and do not use CalledProcessError handling to return True or False. Just checking for returncode value is simpler.
* live: rewrite setup operationv1.2.5Jose M. Guisado2023-02-091-16/+44
| | | | | | | | | | | | | | | | Rewrites the setup operation using python-libfdisk module instead of an external bash script. Consolidating the operation into Python's code, limiting external subprocesses to well known programs and small concrete tasks that are difficult to fully integrate into Python. Use parttypes.py to fetch partition types from python-libfdisk module. Use fs.py to create any specified supported filesystem. OpenGnsys cache partitions are created labelling the partition as "CACHE". Stops setting non-standard MBR hexcode (0xca) to the cache partition in addition to the filesystem label. Any partition specified as type EMPTY will be ignored.
* utils: add init_cache functionJose M. Guisado2023-02-081-0/+12
| | | | | | | | | | | | | | | init_cache() creates the default directory in which OpenGnsys stores images when using any cache enabled transfer method. As of this commit this folder must exist for tiptorrent.py to work properly. Subprocess Popen object inside tiptorrent.py use 'cwd' optional parameter like: cwd='/opt/opengnsys/cache/opt/opengnsys/images/' This folder convention might change in the future.
* utils: add fs.pyJose M. Guisado2023-02-081-1/+66
| | | | | | | | | | | Adds utility module which wraps several mkfs.* calls as a subprocess. The main utility function is mkfs(fs, disk, partition, label), which subsequently calls the corresponding mkfs_*(partition_device) function. mkfs() supports specifying a drive label where supported. Other modules using fs.py should call mkfs() only.
* live: add parttypes.pyJose M. Guisado2023-02-081-0/+55
| | | | | | | | | | | | | Adds parttypes.py module with utility functions to get partition types (parttypes) from python-libfdisk. Supports standard partition types, either DOS or GPT. DOS labels use a hex code to define partition types, python-libfdisk exposes get_parttype_from_code to look up for DOS partition types from a given hexcode. GPT label uses a string (UUID) for each supported partition type, python-libfdisk exposes get_parttype_from_string to look up for GPT partition types from a given string.
* live: clear ogbrowser log before image_createv1.2.4Jose M. Guisado2022-12-051-0/+1
| | | | | Remove any previous unrelated log message shown in the "real time log" html page before executing this command.
* log: add file handler for ogLive "real time log"Jose M. Guisado2022-12-051-0/+9
| | | | | | | | | | | Clients running in ogLive can show log messages via a lighttp server. Particularly, a html page named "real time log" consists of <text-area> tags with the contents of two particular text files /tmp/session.log and /tmp/command.log Adds a Python logging handler in order to write ogClient log messages into /tmp/session.log. This way ogClient logs are show in the "real time log" html page too.
* live: clear ogbrowser logs before image_restoreJose M. Guisado2022-12-011-1/+7
| | | | | | | | | | Clears content of blue text areas in the real time log view before executing a restore image operation. Adds private function _ogbrowser_clear_logs, this function writes to a couple of text files present in the ogLive environment. The contents of this file are printed out to the blue text areas in the "real time log" view.
* live: improve error pathsv1.2.3Jose M. Guisado2022-11-172-5/+22
| | | | | | | | | | | | | Fix error paths in live operations which do not reset the "browser" to the main page (one with the menu). Add error logging messages when: * _restartBrowser fails. * ogChangeRepo fails. Improve checksum fetch error handling. For example, when an invalid repository IP is specified.
* log: declare samba handler only in live modeJavier Sánchez Parra2022-11-021-9/+20
| | | | | | ogClient in linux mode only write its logs to syslog and the console. Also, raise exception if mode its not supported.
* log: import fcntl only on linuxJavier Sánchez Parra2022-11-021-67/+73
| | | | | Otherwise, ogClient do not work on Windows because fcntl module do not exists on Windows.
* live: support native unicast cache image restoreJose M. Guisado2022-11-021-3/+32
| | | | | | | | | | | | | UNICAST-CACHE consist of: 1. Checking if the target image is already present at the opengnsys cache partition. If so, check for integrity (local and remote checksum). If the image is not present in the cache partition, download the target image into it. 2. Restore the image from cache partition. This commit add support for this operation natively from ogClient Python's code.
* tiptorrent: improve raised exceptions for tip_client_getJose M. Guisado2022-09-291-4/+2
| | | | | | Raise exception when tiptorrent-client subprocess runs normally but exits with non-zero code. (For example, if download file allocation failed)
* live: don't use python open() when reading imageJose M. Guisado2022-09-231-5/+3
| | | | | | | | Specifies the image path in the lzop subprocess string. It might be interesting to study efficient mechanisms to read large binary files in python before using open() with default parameters for buffered binary reading.
* live: rewrite image_restoreJose M. Guisado2022-09-143-21/+99
| | | | | | | | Integrates image restore command into native ogClient code. Further reduces the need for external Bash scripts. After a succesful image restore, OS configuration is still using external Bash script "osConfigure/osConfigureCustom".
* utils: add tiptorrent.pyJose M. Guisado2022-09-141-0/+90
| | | | | Utility and wrapper functions related to the usage of tiptorrent. To be used by the image restore command.
* utils: add get_partition_deviceJose M. Guisado2022-09-141-0/+17
| | | | | Maps a disk and partition number to the corresponding device path ('/dev/sdXY', '/dev/nvmeXnYpZ', ...)
* image_restore: fix ogCopyEfiBootLoaderJose M. Guisado2022-09-142-4/+5
| | | | | | | | | ogCopyEfiBootloader is an invalid legacy bash function name. Rename to the correct function name 'ogCopyEfiBootLoader' and rename utility python wrapper too. Fixes: 0bd037c1a409c65fbcb01355ee0dd6dca770330e
* utils: minor fix for ogReduceFs and ogExtendFsJose M. Guisado2022-09-142-12/+11
| | | | | | | | | | Do not return the subprocess result for ogReduceFs/ogExtendFs. ogReduceFs works with or without the target filesystem mounted. ogExtendFs requires the target filesystem to be mounted. 'ogMount' legacy script invocation should be replaced by a better mount/umount wrapper.
* image_create: add legacy ogCopyEfiBootloaderJose M. Guisado2022-09-142-0/+11
| | | | | | | | | | | | | | Use legacy script that saves the Windows-specific content from the ESP to the image target filesystem. Current image restore solution from OpenGnsys scripts expect the EFI partition to be stored in the target system partition. (Only for Windows 10) For example, storing the ESP in the NTFS partition of a Windows image. Expect use of bash script ogCopyEfiBootloader until further integration is merged.
* image_create: partial integration into pythonv1.2.2Jose M. Guisado2022-08-243-26/+162
| | | | | | | | | | | Integrates some parts of this operation into native code, eg: the md5 checksum computation. Wraps non native processes and commands using the subprocess module. For example, legacy.py stores bash commands pending integration. Supports python >=3.6, expected until more modern ogLives are put into production environments.
* refresh: use '0' if disk has no labelJose M. Guisado2022-08-221-1/+4
| | | | | | | | If disk has no label, python-libfdisk returns None when accessing cxt.label Fixes bug when running refresh on computers whose disk/s have no label.
* live: enable details for libfdisk context objectv1.2.1Jose M. Guisado2022-06-081-1/+1
| | | | | | | Enables details in libfidsk context constructor call. Ensures size are displayed as bytes (and not as a human readable string). Avoids further arithmetical errors (e.g: converting from bytes to KB)
* live: remove unused parseGetConf functionJose M. Guisado2022-06-081-27/+0
| | | | | | parseGetConf was necessary for parsing output from OpenGnsys script "getConfiguration", which has been replaced since commit 1ab981a539f3 ("live: replace getConfiguration with refresh")
* src: improve loggingJose M. Guisado2022-06-089-27/+93
| | | | | | | | | | | | | | | | | | | | | | | | | | | Adds new logging handler redirecting messages to the log file located in the Samba shared directory (applies to live mode clients, i.e: ogLive) Parses log level configuration from ogclient.json. See: { "opengnsys": { ... "log": "INFO", ... } ... } Adds --debug option to set root logger level to DEBUG when starting ogClient. Overrides log level from config file. In addition: - Replaces any occurence of print with a corresponding logging function. - Unsets log level for handlers, use root logger level instead. - Default level for root logger is INFO. - Replaces level from response log messages to debug (ogRest)
* live: replace getConfiguration with refreshJose M. Guisado2022-06-011-10/+4
| | | | | | | Old refresh was a wrapper around getConfiguration bash script. New refresh operation does what getConfiguration used to do externally. See commit 097769b971f64f93004a87393721a289fe3db13f.
* live: assume VFAT is always FAT32Javier Sánchez Parra2022-06-011-0/+3
| | | | | fdisk reports VFAT, however, OpenGnsys does not have such entry in the filesystem table. Add an alias to FAT32 as a workaround.
* utils: mount_mkdir success if target is a mountpointJose M. Guisado2022-06-011-0/+6
| | | | | | | | Returns true if target is already a mountpoint. Does not call mount. It's possible that another device might be mounted in the target mountpoint. A future check between the source and target for equal device major:minor must be added.
* live: generate cache.txt file in refreshJose M. Guisado2022-06-012-0/+71
| | | | | | | | | | | | | Generates a cache.txt file if a cache partition is detected. OpenGnsys stores information about stored images in its 'cache' partition via a text file. The file is stored in a samba shared directory, mounted at '/opt/opengnsys/log/' in a live client. The file name is '{ip}.cache.txt'. Previously, the generation of this file was delegated to external bash scripts.
* probe: detect 64 bit operating systemsJose M. Guisado2022-05-092-2/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | OpenGnsys partition images store OS information, including the OS name appended with "64 bits" when the OS is meant for 64 bit machines. The detected OS name when probing (refresh) is important, if it differs from what's stored in the DB OpenGnsys will wipe last image restored information when running a refresh. See actualizaConfiguracion from legacy ogserver (ogAdmServer.c) code: dato = dbi_result_get_uint(result, "idnombreso"); if (idsoi == dato) { swu = false; } ... if (swu) { result_update = dbi_conn_queryf(dbi->conn, "UPDATE ordenadores_particiones SET " " codpar=0x%s," " tamano=%s," " uso=%s," " idsistemafichero=%d," " idnombreso=%d," " idimagen=0," " idperfilsoft=0," " fechadespliegue=NULL" " WHERE idordenador=%d AND numdisk=%s AND numpar=%s",
* Add link speed to refresh responseJavier Sánchez Parra2022-05-091-1/+4
| | | | | Probe response already has client's link speed, but this API is deprecated.
* utils: fix winreghives path stringJose M. Guisado2022-04-281-1/+1
| | | | | | | Adds missing slash in the windows reghives path f-string. When calling os_probe inside refresh (live/ogOperations) the mountpoint is passed without a trailing slash.
* live: detect cache partitions in new refreshJose M. Guisado2022-04-282-3/+23
| | | | | | | Detects OpenGnsys cache partition when building the partition setup of a live system. OpenGnsys labels a Linux/ext4 partition as "CACHE".
* live: get partition number using partno attributeJose M. Guisado2022-04-271-6/+7
| | | | | | | | | | | | Removes a workaround enumaration of the partitions of a given libfdisk context. Use Partition class partno attribute. This enables detecting non contiguous partitions, like: /dev/sda /dev/sda1 /dev/sda2 /dev/sda4
* utils: decode probing subprocess stdoutJose M. Guisado2022-04-271-2/+2
| | | | | | Cannot replace str ('\n') when stdout is a bytes object. Also fixes some typos (i.e: proc_releaseid stdout was used twice).
* live: use utils.py and fdisk in refresh operationJose M. Guisado2022-04-211-12/+71
| | | | | | | | | | | | Use python binding for libfdisk to retrieve disk label and partition information [1]. Use utils.py for the rest of the process. This change aims to drop dependency with external script 'getConfiguration' from the OpenGnsys scripting ecosystem. Keeps the same json payload structure for the refresh response. [1] https://git.48k.eu/python-libfdisk/
* live: use utils.py for probe operationJose M. Guisado2022-04-211-22/+4
| | | | Import ethtool function from utils.py
* Add partcodes.py to src/liveJose M. Guisado2022-04-211-0/+25
| | | | | | | | | | | | partcodes.py is used for mapping GPT partition UIIDs to hexcode/decimal values. OpenGnsys stores partition types in the database using the decimal value of this hexcode. The hexcode used for GPT partition UUID is the same as defined internally in gdisk [1]. [1] https://sourceforge.net/p/gptfdisk/code/ci/master/tree/parttypes.cc#l75
* Add utils modulesJose M. Guisado2022-04-215-0/+273
| | | | | | | | | | | | | | | | | | | | | | | | * disk.py Disk discovery * fs.py Uses psutil to fetch fs usage information * menu.py ogBrowser menu generation * net.py: gets nic status information IP address, MAC address and ethernet speed. * probe.py: probes mountpoints for operating systems Uses hivexget command to try fetching Windows installation information. Looks for /etc/os-release for probing linux systems.
* Revert "#1065 linux: add systray icon"Jose M. Guisado2022-02-031-53/+0
| | | | This reverts commit 69d214f63b2aa8ef60489d56468393b70795124a.
* #1065 Init event socket according to modeJose M. Guisado2022-02-021-1/+3
| | | | | | | Commit 700aa89ea99 introduced the use of getattr when getting event socket to avoid errors in case it was not initialized. Prefer to initialize accordingly inside the ogClient class constructor.