summaryrefslogtreecommitdiffstats
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
...
* utils: add uefi.pyJose M. Guisado2024-03-041-0/+97
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add UEFI related utilities inside a new utility module: uefi.py _check_efibootmgr_json ====================== Check if the system efibootmgr executable supports JSON output. This is a private function used only by other functions from uefi.py. is_uefi_supported ================= Check if the system supports UEFI firmware. run_efibootmgr_json =================== Runs efibootmgr with json output support. Return the JSON output as a Python dict. efibootmgr_create_bootentry =========================== Create nvram boot entry. This bootentry is usually later set to boot next just once via "BootNext" nvram variable. efibootmgr_delete_bootentry =========================== Delete a nvram boot entry. Used to avoid duplicates when booting the same disk and partition from a given client. efibootmgr_bootnext =================== Set nvram "BootNext" variable to a given boot entry so after client reboot, PXE is not executed and the given boot entry takes precedence. Add dependency with efibootmgr version >= 18, and efibootmgr JSON output which is currently out of tree from util-linux repo.
* probe: add basic os family detectionJose M. Guisado2024-03-041-0/+16
| | | | | | | | Add a basic OS family enumeration: OSFamily. Add utility function that probes for an installed Linux or Windows system, returns the corresponding enum value, OSFamily.UNKNOWN otherwise.
* disk: add get_efi_partition functionJose M. Guisado2024-03-041-0/+24
| | | | | | | | | | | | | Add utility function inside disk.py to find, if any, the first ESP partition of a given disk. The disk is provided as an integer (starting at 1 following OpenGnsys scripts usual values), meaning the (n-1)th disk from the disk array returned from get_disks(). In the future a better mechanism should be put in place to fetch probed disks from a running client. This change is part of the upcoming drop of "IniciarSesion" script in favor of a Python native approach. Specifically regarding UEFI systems.
* cache: improve loggingOpenGnSys Support Team2024-02-221-1/+1
| | | | | use info instead of debug to make it easier to debug problems when creating the cache.
* live: improve logging with setup commandOpenGnSys Support Team2024-02-221-2/+10
| | | | | | Improve logging when setting up partition, provide more hints on progress. Fail in case partition layout is not supported.
* fs: add swap support and improve mkfs loggingOpenGnSys Support Team2024-02-221-3/+15
|
* utils: use returncode from subprocess in ogChangeRepo()OpenGnSys Support Team2024-02-211-4/+3
| | | | | | | ... the exception shows the samba password in the logs specify the error which tells us what has happened according to man mount(8) Return Codes.
* live: call partprobe on the specific diskOpenGnSys Support Team2024-02-191-1/+1
| | | | | otherwise partprobe does its best to find the disk, according to what I see through strace.
* poweroff: always call poweroff_oglive and _reboot_ogliveOpenGnSys Support Team2024-02-191-8/+2
| | | | Remove leftover fallback to directly call utilities to poweroff and reboot.
* fs: improve value parsing from command outputAlejandro Sirgo Rica2024-02-191-7/+29
| | | | | | value extraction did not have error checking and was handled in a one-liner. The actual implementation expands the parsing logic and moves it into a function.
* live: split logging to warn not to turn off client during image creationOpenGnSys Support Team2024-02-151-1/+2
| | | | just split this log message.
* utils: remove repository change in restoreImageCustomOpenGnSys Support Team2024-02-151-4/+0
| | | | this is broken, it uses default uses and password, remove it.
* fs: check if writing md5sum to full.sum file failsOpenGnSys Support Team2024-02-151-3/+11
| | | | writing to file might fail (permission denied, disk full), check for errors.
* fs: return unknown if blkid failsOpenGnSys Support Team2024-02-151-1/+7
| | | | instead of rising an exception
* fs: logging ext4 resize errorOpenGnSys Support Team2024-02-151-2/+4
| | | | log error in case resize2fs fails.
* fs: check if ntfsresize actually succeded to shrink filesystemOpenGnSys Support Team2024-02-151-1/+4
| | | | | According to ntfsresize.c, this retuns 0 in case nothing needs to be done. It should be safe to check for non-zero error and bail out in that case.
* fs: disentagle dry-run ntfsresize loop to probe for best shrink sizeOpenGnSys Support Team2024-02-151-8/+16
| | | | | | | | Revisit 5056b8f0d5ab ("fs: validate ntfsresize dry-run output") that has introduced a possible infinity loop. Disentangle this loop while at it: iterate until best smallest size is found by probing.
* src: ogChangeRepo returns zero on success and -1 on errorOpenGnSys Support Team2024-02-152-9/+12
| | | | | | | | | | | | | | | | | | | | do not return the returncode, instead return an integer. do not use except CalledProcessError as e: it causes a another exception while handling exception. Remount the original image repository. it should be possible to simplify this further by: - stacking mounts, no need to umount initial repo and mount it again when switching to the new repo, because remount back initial repo might fail (!) - use check=False and simply check for x.returncode
* live: remove mbuffer leftover in image restore commandOpenGnSys Support Team2024-02-151-1/+0
| | | | | | | | | | | | | | | | | | | Remove mbuffer, this is never used. mbuffer has been never been used since ogClient supports native image restore. Originally this was used like this: partclone [...] | mbuffer -q -M 40M | lzop [...] supposely to speed up partclone in case the device where the read happens is slowier than the device that is used for writes. See mbuffer(1) manpage examples. In any case, this needs benchmarking to really make sure this is helping. Remove it until that ever happens.
* live: log message improvements for image creation and restoreOpenGnSys Support Team2024-02-151-6/+12
| | | | | Provide more context information for debugging issues with image creation and restore.
* src: improve error check in image_create and image_restoreAlejandro Sirgo Rica2024-02-144-11/+29
| | | | | | | | | cover more error cases where exceptions need to be raised. check return code in the invoked subprocess. restoreImageCustom has been intentionally left behind, it is unclear what this custom script returns on success and error.
* src: fix whitespace in ogOperations.pyAlejandro Sirgo Rica2024-02-141-13/+13
| | | | make whitespace conherent with the rest of the file contents.
* fs: validate ntfsresize dry-run outputv1.3.2-5OpenGnSys Support Team2024-01-111-2/+3
| | | | | | | | | | | | | | | | validate 'Needed relocations: ' is in place before stepping on the split chunks (2024-01-11 10:28:16) ogClient: [ERROR] - Exception when running "image create" subprocess Traceback (most recent call last): File "/opt/opengnsys/ogClient/src/live/ogOperations.py", line 454, in image_create ogReduceFs(disk, partition) File "/opt/opengnsys/ogClient/src/utils/fs.py", line 105, in ogReduceFs _reduce_ntfsresize(partdev) File "/opt/opengnsys/ogClient/src/utils/fs.py", line 235, in _reduce_ntfsresize extra_size = int(out_resize_dryrun.split('Needed relocations : ')[1].split(' ')[0])*1.1+1024 IndexError: list index out of range if not present, no need to adjust size
* live: refine existing loggingOpenGnSys Support Team2023-12-181-2/+2
| | | | | - suggest to check permissions in samba folder - fix typo, s/filesyste/filesystem/
* live: adding logging to notify that image file already existsv1.3.2-4OpenGnSys Support Team2023-12-171-0/+3
| | | | Just informational, provide a notice that the file already exists.
* live: ensure image file exists after partcloneOpenGnSys Support Team2023-12-171-0/+4
| | | | check that there is a file and that is accessible
* live: display filesystem and device path if image_create() failsOpenGnSys Support Team2023-12-161-1/+1
| | | | display filesystem and path to device.
* live: validate rw access to image folder after remountOpenGnSys Support Team2023-12-161-0/+5
| | | | check that it is readable and writable
* utils: use f-string in raise ValueError in run_lzop_partcloneinfo()OpenGnSys Support Team2023-12-151-1/+1
| | | | | | Otherwise it shows: ValueError: Unable to process image {image_path}
* live: report permissions and last update when creating imagev1.3.2-3OpenGnSys Support Team2023-12-123-3/+14
| | | | add .permissions and .lastupdate to json to report to ogserver.
* live: report image size when creating imageOpenGnSys Support Team2023-12-123-0/+11
| | | | add .size json field to report the real size of the image file.
* utils: sw_inventory: report unknown OS for software inventoryv1.3.2-2OpenGnSys Support Team2023-11-171-1/+2
| | | | | | Users can create an image of a filesystem that contains no OS, therefore, instead of rising an exception when no OS is detected, deliver a "unknown" OS and an empty list of software.
* live: use legacy backup image suffixv1.3.2Jose M. Guisado2023-10-241-2/+1
| | | | | | | | | | | Image backup is considered a legacy feature. Use the legacy mechanism of naming image backups by adding ".ant" suffix. Previously, by using the strftime suffix clients were reporting that the disk were getting full rather quickly. When a good method for image deletion is implemented then a proper backup naming mechanism should be reconsidered.
* live: hw_inventory: fix empty pci storage size bugJose M. Guisado2023-10-181-1/+1
| | | | | | | | | | | When a client's hardware presents an empty pci storage child there is an invalid call to _bytes_to_human a string is supplied as a default value if the storage child does not present a 'size' attribute. Fix this by checking if 'size' is present in the JSON output from lshw. If size is present then map the bytes to a human readable string using _bytes_to_human, if no size is present then use 'Empty slot' to indicate that the memory bank is not being used.
* live: hw_inventory: fix typoJose M. Guisado2023-10-181-1/+1
| | | | | | Add missing underscore to _bytes_to_human call. Fixes: 39c13287c53bd8 ("live: hw_inventory: fix empty memory bank bug")
* live: hw_inventory: fix empty memory bank bugJose M. Guisado2023-10-051-6/+9
| | | | | | | | | | | | When a client's hardware presents an empty memory bank and invalid call to _bytes_to_human is performed because None is passed as a parameter. size = _bytes_to_human(obj.get('size', None)) Fix this by checking if 'size' is present in the JSON output from lshw. If size is present then map the bytes to a human readable string using _bytes_to_human, if no size is present then use 'Empty slot' to indicate that the memory bank is not being used.
* src: improve logging messagesJose M. Guisado2023-08-012-9/+15
| | | | | | | | | | | | | | | | | | | | | | | | | Some users have mistakenly reported tiptorrent problems when the process takes a long time. Specifically by rebooting or powering off the client in the middle of the md5sum computation stage, just after the tiptorrent transfer. Same problem occurs when image creation command takes a long period of time. In order to help the user understand the different stages of commands such as image creation or image restore using tiptorrent, the following changes have been made to the current logging solution: - Add log messages to warn users not to reboot or shut down the client during a tiptorrent transfer, and also during the md5sum computation stage. - Add a log message telling the user that the image creation processes have started. - Use logging.exception inside "except:" blocks to print a traceback with the log messsage. (https://docs.python.org/3/library/logging.html#logging.exception)
* hw_inventory: use dict.getJose M. Guisado2023-07-211-23/+27
| | | | | | | | | The first stage of parsing the "lshw -json" command output is to load the json string into a Python dictionary. lshw output is large and varies from machine to machine, so it's not safe to assume that different keys will be present in the dictionary. Use dict.get() instead of dict[key] to avoid KeyError exceptions.
* live: add image backup option in image creationv1.3.1Jose M. Guisado2023-07-061-0/+6
| | | | | | | | | | | | Backup image file if image creation request included "backup": true This only applies when the target image is already present in the repository folder before running the partclone subprocess. This parameter is ignored if the target image is not present in the repository.
* rest: add backup parameterJose M. Guisado2023-07-061-0/+7
| | | | | | | Support parsing of "backup" parameter inside incoming requests. Prepare ogClient support for backup image creation in "image create" operation.
* ogclient: add support for X-Sequence headerv1.3.0Jose M. Guisado2023-06-143-22/+35
| | | | | | | | | | | | | | | Enable parsing of "X-Sequence" HTTP headers from incoming requests. Add "seq" field in restRequest class. Enable adding "X-Sequence" to outgoing responses. Add "seq" field inside restResponse class. Store current client sequence number inside ogClient class. Ideally, the restRequest object should be used to retrieve the sequence number but not all processing functions inside ogRest.py receive the request as parameter (eg: process_refresh). In the other hand, all processing functions receive the ogClient object.
* fs: fix subprocess input inside _extend_resize2fsJose M. Guisado2023-06-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | The subprocess module expects bytes-like object for "input" parameter by default. Passing a string object result in the following error: (2023-06-13 14:44:43) ogClient: [ERROR] - Exception when running "image create" subprocess (2023-06-13 14:44:43) ogClient: [ERROR] - Unexpected error Traceback (most recent call last): File "/opt/opengnsys/ogClient/src/live/ogOperations.py", line 465, in image_create ogExtendFs(disk, partition) File "/opt/opengnsys/ogClient/src/utils/fs.py", line 124, in ogExtendFs _extend_ntfsresize(partdev) File "/opt/opengnsys/ogClient/src/utils/fs.py", line 250, in _extend_ntfsresize proc = subprocess.run(cmd, input='y') File "/usr/lib/python3.8/subprocess.py", line 495, in run stdout, stderr = process.communicate(input, timeout=timeout) File "/usr/lib/python3.8/subprocess.py", line 1013, in communicate self._stdin_write(input) File "/usr/lib/python3.8/subprocess.py", line 962, in _stdin_write self.stdin.write(input) TypeError: a bytes-like object is required, not 'str' Fixes: dd999bfe34e7 ("utils: rewrite ogReduceFs")
* fs: fix bug when ntfsresize reports nothing to doJose M. Guisado2023-06-131-0/+3
| | | | | | | | There is a corner case in which a target NTFS filesystem is already shrunken. When this happens ntfsresize text output parsing breaks. Check when ntfsresize reports nothing to do, warn the user about this and stop the dry-run ntfsresize loop.
* fs: fix typo inside _extend_resize2fsJose M. Guisado2023-06-131-4/+4
| | | | | | | | | | _extend_ntfsresize contains an incorrect variable name inside subprocess.run referring the resize command value. Simplify this variable name inside each specific _extend_* function: s/cmd_resize2fs/cmd s/cmd_ntfsresize/cmd
* src: remove unused legacy software inventory codeJose M. Guisado2023-05-172-26/+9
| | | | | | | | | | | | | Remove unnecessary InventarioSoftware invocation inside image_create operation. Software inventory is executed after image creation (see ogRest.py). Remove legacy 'path' parameter. This parameter was used to specify the path of a text file in which legacy bash scripts wrote the software inventory of the client (something like "Csft-{ip}..."). Fixes: 04bb35bd86b58c ("live: rewrite software inventory") Fixes: 2e3d47b7b8db69 ("Avoid writting /software output to a file")
* sw_inventory: ignore invalid windows programsJose M. Guisado2023-05-091-5/+10
| | | | | | | | | | | | | | | | | | | | | | | Don't raise exception if any windows program is missing DisplayName node in the windows registry. This attribute/node should contain the program's name. This name is used as the package's name in the software set (software inventory). This patch should be considered a hotfix, python-hivex does not report any helpful message about this error. (2023-05-09 14:43:13) ogClient: [ERROR] - Unexpected error Traceback (most recent call last): [...] RuntimeError: Success Before this patch, image creation *might* fail because it cannot create the software inventory associated with the image due to the previously described error. The software inventory is part of the response payload of the image creation command (see src/ogRest:image_create). Fixes: 04bb35bd86b5 (live: rewrite software inventory)
* live: rewrite reboot operationv1.2.9Jose M. Guisado2023-05-091-5/+3
| | | | | | Add optional 'operation' parameter to _poweroff_oglive function. Reuse _poweroff_oglive code before the busybox subprocess when rebooting an ogLive client.
* live: rewrite poweroff operationJose M. Guisado2023-05-091-3/+20
| | | | | | | | | | | | | Replace legacy bash script /opt/opengnsys/client/scripts/poweroff with a Python native solution. Use subprocess module for any required external program when shutting down a client. ethtool is used to ensure WoL setting is correct before shutting down. ogLive does not properly use a init system so busybox is used when shutting down the system. In other live environments poweroff operation just calls /sbin/poweroff.
* utils: add umount_all functionJose M. Guisado2023-05-091-0/+9
| | | | | | | | Add utility function to unmount any mountpoint present in the /mnt folder. This function is a simplified version of the legacy bash function ogUnmountAll used in several operations.
* utils: rewrite ogExtendFsJose M. Guisado2023-05-091-7/+28
| | | | | | | | | | | | | | | | | | | | Drop subprocess call to bash function ogExtendFs. Use a native python solution with subprocess calls to the required underlying tools. Use get_filesystem_type to get the present filesystem from a partition and call the corresponding filesystem grow function. Filesystem specific functions are declared "_extend_{filesystem}" and should not be imported elsewhere. Each filesystem specific function wraps a subprocess call to the required underlying program: - NTFS filesystems: "ntfsresize -f [partition]" - ext4 filesystems: "resize2fs -f [partition]" Set NTFS related subprocess stdin to 'y' because human input cannot be unset with other ntfsresize parameters.