summaryrefslogtreecommitdiffstats
path: root/src/utils
Commit message (Collapse)AuthorAgeFilesLines
* utils: add set_windows_hostnameAlejandro Sirgo Rica2024-07-291-0/+47
| | | | | | | | | | | | | | | | | | | | | | | Add function to redefine the hostname of a Windows install. Windows hostnames can't be larger than 15 characters due to legacy heritage. Hostname modification is done by modifying 3 registry values in the ControlSetXXX entry of the SYSTEM hive. ControlSet001 is generally the entry to be edited but one must query the value of the 'Current' key in the 'Select' entry of the SYSTEM hive to retrieve the active ControlSet. The hostname has to be introduced in the following entries: path = 'ControlSetXX/Control/ComputerName/ComputerName' key = 'ComputerName' path = 'ControlSetXXX/Services/Tcpip/Parameters' key = 'HostName' key = 'NV Hostname' The value to store in those keys is of tipe SZ and has to be encoded in UCS-2 Little Endian (utf-16-le).
* utils: add postinstall.pyAlejandro Sirgo Rica2024-07-292-21/+170
| | | | | | | | | | | | | | | Replace ConfigureOs script by native calls to: - update BCD via hivex using bcd.py and winreg.py infrastructure. - restore efi bootloader restore_windows_efi_bootloader(). Call legacy scripts for remaining postinstall procedures to replace them incrementally. Define variable CONFIGUREOS_LEGACY_ENABLED as False by default. Run legacy configureOs when CONFIGUREOS_LEGACY_ENABLED = True. This serves as a auxiliar method to keep the restoration functional in case of problems with the new configure_os logic.
* utils: add BCD native supportAlejandro Sirgo Rica2024-07-291-0/+198
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement update_bcd() as replacement of ogWindowsBootParameters. The legacy function modified the BCD of a freshly restored system invoking the privative binary "spartlnx". The script edits a set of BCD entries needed for a proper system boot. Each main BCD entry is identified by an UUID and contain a set of subnodes, these hold the configuration and entry information. Each node contains data in the form of key-value. Common BCD structure: 'Objects' ... The Boot Manager entry is always identified by the UUID 9dea862c-5cdd-4e70-acc1-f32b344d4795. Some entries always have the same UUID as identifier such the Boot Manager while other have different UUID depending on the system. To identify these entries with a not known UUID we query the value 'Type' of the node 'Description'. This contains a 32 bit value whose bytes codify the nature of the entry. We obtain 3 different values as a tuple, each value is the result of applying a bitmask to the Type value. These masks are 0xf0000000, 0x00f00000 and 0x000fffff. The resulting tuple of 3 values is then used to obtain the corresponding entry from a map. The data we modify from the BCD are disk and partition references to point to the new disk and partition in the system where the images are restored. Partition and disk information is stored as UUID in specific offsets inside binary data in the BCD. To update these we need to obtain the disk and partition UUIDs, then convert it to bytes as follows: original UUID: C4C61C51-3456-4733-96AD-AE760A41C392 UUID as bytes: 51 1C C6 C4 56 34 33 47 96 AD AE 76 0A 41 C3 92 The modified entries are: Resume from Hibernation, Windows Boot Loader OS, Windows Boot Loader Recovery, Windows Recovery, Boot Loader Settings, Windows Boot Manager and Windows Memory Diagnostic. Some of these options could be omited as the system restoration does not include a recovery partition so in this case all the recovery related entries just point to the main system partition. Most entries are edited modifying the value corresponding to the key 'Element' in 2 subnodes of the 'Elements' node. These subnodes are '11000001' and '21000001'. 'Objects' The 2 values stored in these 2 entries is binary data where we store the partition and disk bytes. We simply replace the byte representation of out UUIDS in the binary data. Partition is stored in offset 32 and the disk in offset 56. The exception is the Bootloader Recovery entry, in which the partition offset is 84 and the disk offset 108. Note that the legacy function only does a proper BCD edit in UEFI systems. The new implementation follows the same behavior with the possibility of implementing BCD modification under MBR partitions in the future. Set the field 16000009 (RECOVERY_ENABLED) to the value x00 to disable recovery in both Resume from Hibernation and Windows Boot Loader OS entries in the BCD. The system install does not include any recovery partition so it makes no sense to have it enabled. This commit is preparatory work for the new native postinstall code.
* utils: add win edit registry utilitiesAlejandro Sirgo Rica2024-07-291-0/+114
| | | | | | | | | | | | | | | | | Add winreg.py to the utils folder. Implement hive enum types, hive handler validation and validated get_* functions for nodes and registry values. Implement the utility hive operations through the Hivex library. This serves as preparatory work for BCD manipulation but it also has potential to improve registry usage in previous code. UCS-2 Little Endian is the prefered windows registry text encoding for binary content. Define a WINDOWS_HIVE_ENCODING global variable to use when encoding string to write in the win registry. This commit is preparatory work for the new native postinstall code.
* utils: add restore_windows_efi_bootloaderAlejandro Sirgo Rica2024-07-291-0/+31
| | | | | | | Add restore_windows_efi_bootloader to copy the EFI loader from the filesystem in the restored system into the EFI partition. This commit is preparatory work for the new native postinstall code.
* utils: add functions to obtain disk and partition UUIDsAlejandro Sirgo Rica2024-07-291-0/+25
| | | | | | | | Add get_partition_id and get_disk_id to obtain the UUID of a disk or partition as a string. These values are obtained invoking the program blkid as a subprocess. This commit is preparatory work for the new native postinstall code.
* utils: consolidate code to find efi loaderAlejandro Sirgo Rica2024-07-292-30/+29
| | | | | Add find_windows_efi_loader and find_linux_efi_loader to reduce code duplication and to centralize efi loader path modifications.
* utils: rename ogExtendFs to extend_filesystemAlejandro Sirgo Rica2024-07-291-1/+1
| | | | Use a more consistent name format for the ogExtendFs function.
* utils: fs: error out if mkfs failsOpenGnSys Support Team2024-07-291-5/+17
| | | | | | | | | | If formatting fails, log shows: mkfs.ntfs reports return code 1 for /dev/sda2 but ogclient reports success to the ogserver. Raise an exception so front-end gets an indication that formatting has failed.
* utils: consolidate logging to write checksum file to cacheOpenGnSys Support Team2024-07-211-2/+3
| | | | Move log message to function that is called both by unicast and tiptorrent.
* utils: missing f-string with filesystem resize errorv1.3.2-13OpenGnSys Support Team2024-07-151-1/+1
| | | | ... failed to resize {partdev} with ext4
* utils: clean up error reporting related to checksumOpenGnSys Support Team2024-07-151-2/+2
| | | | | | | | When checksum is not available, it displays: (2024-07-15 09:04:14) ogClient: [ERROR] - URL error when fetching checksum: Not Found "URL error" is leaking an internal implementation details, reword this report.
* live: add checksum field to image/create responseOpenGnSys Support Team2024-07-151-0/+1
| | | | Report image checksum to ogserver through HTTP response.
* live: revisit error log when failing to validate checksumOpenGnSys Support Team2024-06-271-1/+4
| | | | | | | | Add explicit check for .full.sum after downloading it. Rewrite errors log, one of them is misleading when checksum validation fails, it refers to missing .full.sum, but it could be a different reason.
* utils: remove bogus cache not found error after refresh commandAlejandro Sirgo Rica2024-06-071-1/+0
| | | | | | Remove log message "Cannot find device path to cache" spamming logs during operations such as refresh in clients without CACHE partition.
* utils: fix get_image_info regressionv1.3.2-10Alejandro Sirgo Rica2024-06-031-5/+6
| | | | | | | Fix image size, permissions and creation time. Improve error report related to these parameters now showing the exact cause of the problem if any occurred during the definition of image size, file permissions or image creation time values.
* src: cleanup ogGetImageInfoAlejandro Sirgo Rica2024-05-301-2/+11
| | | | | | Rename ogGetImageInfo to get_image_info. Move code from ogOperations.py to obtain image data into get_image_info.
* src: rename legacy.py into image.pyAlejandro Sirgo Rica2024-05-301-0/+0
| | | | | legacy.py contais mostly functions related to system images. Rename the file to better represent the contents in it.
* src: stop using hardcoded paths to cache image directoryAlejandro Sirgo Rica2024-05-301-4/+4
| | | | | | Use the constant OG_CACHE_IMAGE_PATH from cache.py to obtain the location of the directory where images are stored. This way the path can be changed from one single point.
* tiptorrent: check cache availability in tip_write_csumAlejandro Sirgo Rica2024-05-301-3/+4
| | | | | | Remove old TODO message. Check if the cache is available before trying to generate an image's checksum.
* utils: cache: redefine cache path constantsAlejandro Sirgo Rica2024-05-271-8/+9
| | | | | | | | | Rename OGIMG as OG_IMAGE_PATH. Rename OGCACHE_MOUNTPOINT as OG_CACHE_PATH. Define OG_CACHE_IMAGE_PATH as OG_CACHE_PATH + OG_IMAGE_PATH. This will serve to have a unique point to obtain cache related paths.
* utils: rename cache_probe() to get_cache_dev_path()OpenGnSys Support Team2024-05-092-6/+9
| | | | | | | | This method reports the /dev path to cache partition, rename it. Add explicit check if blkid is successful. And add logging to report that device path to cache is not found.
* utils: add error checks to checksum file creation in tip_write_csumAlejandro Sirgo Rica2024-05-071-2/+5
| | | | | Add a check for potential permission or IO errors during the creation of the image checksum.
* utils: add mkdir error report in mount_mkdirAlejandro Sirgo Rica2024-05-071-1/+5
| | | | | | Add exception checks to the os.mkdir operation and log the error found. The previous implementation was too optimistic and only handled mount related errors.
* utils: remove unused json importAlejandro Sirgo Rica2024-05-071-1/+0
| | | | | The json library was a dependency during the development of the boot OS functions and it is no longer needed in that file.
* utils:fs: fix logging statement in get_filesystem_typeAlejandro Sirgo Rica2024-05-071-1/+1
| | | | | Call the error() logging function from the logging object instead of the non existent log variable previously referenced.
* utils:fs: add mkfs logs when return code is not 0Alejandro Sirgo Rica2024-05-071-12/+24
| | | | | | Report mkfs failure for every partition. This does not raise an exception as that would skip partprobe operations and the mkfs operations in the next potentially well formated partitions.
* utils: add logging for checksum validationOpenGnSys Support Team2024-04-221-3/+9
| | | | Display if checksum validation is correct or not.
* utils: fs: incorrect indentationOpenGnSys Support Team2024-04-221-1/+1
|
* utils: fs: skip shrink/extend operation for fatOpenGnSys Support Team2024-04-191-0/+4
| | | | fat does not support this operation, skip it
* utils: fs: remove unused variable in _reduce_resize2fsv1.3.2-7OpenGnSys Support Team2024-04-191-1/+0
| | | | ret is set but never used in this method
* utils: fs: report error when failing to growOpenGnSys Support Team2024-04-191-2/+2
| | | | not really a warning, report an error instead
* utils: fs: set return value in ogReduceFsOpenGnSys Support Team2024-04-191-2/+3
| | | | | | | | otherwise error path uses uninitialized variable File "/opt/opengnsys/ogClient/src/utils/fs.py", line 112, in ogReduceFs return ret UnboundLocalError: local variable 'ret' referenced before assignment
* src: use explicit exception types in except Exception blocksAlejandro Sirgo Rica2024-04-032-3/+3
| | | | | | | | Capture only the relevant exception types in each except block. The capture of the Exception type means hiding information for unhandled error cases, even for syntax errors in the codebase. Using a more fine grained exception filtering improves error traceability.
* src: log backtrace in unhandled error casesAlejandro Sirgo Rica2024-04-038-49/+57
| | | | | | | | | | | | | | | | | | | Log an error message in known error cases and log a backtrace otherwise. Define a new error type OgError to be used in all the 'raise' blocks to define the error message to log. The exception propagates until it reaches send_internal_server_error() where the exception type is checked. If the type is OgError we log the exception message. Logs the backtrace for other types. The initial error implementation printed a backtrace everytime an error ocurred. The next iteration changed it to only print a backtrace in a very particular case but ended up omiting too much information such as syntax errors or unknown error context. The actual implementation only logs the cases we already cover in the codebase and logs a bracktrace in the others, enabling a better debugging experience.
* utils: refine log when booting windows UEFI from DOS partitionOpenGnSys Support Team2024-04-011-1/+1
| | | | | | Refine 97647c32aa2c utils: add enforce_gpt argument to get_efi_partition() to provide more explicit error when trying to boot Windows UEFI from DOS partition.
* utils: fix mount error messages for os probe operationsAlejandro Sirgo Rica2024-04-012-2/+2
| | | | | | | | Replace unexistent mountpoint variable to report a failed mount operation before an OS probe from a partition. Improve the semantics of the error message replacing 'at' with 'into'. Remove the period at the end of the log message.
* utils: make init_cache() use the cache mountpointAlejandro Sirgo Rica2024-03-261-1/+1
| | | | | | Make init_cache() use the actual cache mountpoint returned by the function mount_cache() for the creation of the cache directories instead of a hardcoded path.
* utils: remove redundant return statements from mount_mkdirAlejandro Sirgo Rica2024-03-261-3/+1
| | | | | | Remove return statement the program won't reach and simplify the return logic with a return from an if and a fallback return statement.
* utils: drop ogCopyEfiBootLoader scriptAlejandro Sirgo Rica2024-03-262-10/+52
| | | | | | | | Implement a Python equivalent of ogCopyEfiBootLoader as the function copy_efi_bootloader. This function copies the contents of the folder of the EFI loader in the ESP into a ogBoot folder at the root of the partition target of an image creation. copy_efi_bootloader is a Windows only functionality.
* utils: add enforce_gpt argument to get_efi_partition()Alejandro Sirgo Rica2024-03-262-5/+8
| | | | | | | The Windows bootloader only supports a UEFI boot from a GPT partition. Set enforce_gpt to True in every codepath related to Windows. When enforce_gpt is set to True get_efi_partition() raises an exception when an MBR partition scheme is detected.
* revert 7f18485: improve uefi detection mechanismAlejandro Sirgo Rica2024-03-262-13/+3
| | | | | | | | | | | Make is_uefi_supported() only check for /sys/firmware/efi as get_efi_partition() will detect a missing ESP or an invalid partition scheme. Stop using get_efi_partition() inside is_uefi_supported() as the former is eventually called in every UEFI related code. UEFI supports both MBR and GPT as partition schemes and this is a required change to handle the particular case of Windows not being able to boot UEFI from a MBR partition scheme.
* src: make exception messages more contextual and explicitAlejandro Sirgo Rica2024-03-213-5/+5
| | | | | | | Provide more information in exception messages as those are the source of the logging messages. Add information about paths, files or configuration related to the operation associated to the exception.
* src: change generic exception types to be more explicitAlejandro Sirgo Rica2024-03-211-1/+1
| | | | | | | | Replace exception types to be more explicit about the nature of the error. Improve the exception raising semantics by using the 'from' keyword, this wraps an older exception into a new one so it is still considered the same object.
* utils: fix string not being defined as f-strings when they shouldAlejandro Sirgo Rica2024-03-211-2/+2
| | | | | Add the string prefix f'' to string instances formated as f-string but not properly constructed to expand the inline variables.
* src: centralize error logging into send_internal_server_errorAlejandro Sirgo Rica2024-03-215-41/+28
| | | | | | | | | | | | | | | | | | | | Use only the exception messages as the main resource for error messages. The previous error code had string duplication in the form of: logging.error('msg here') raise Exception('msg here') That approach also has the downside of having log duplication as it had the local logging.err() and a global logging.exception() inside send_internal_server_error capturing the exception message. The actual code only requires raising an exception with a proper error message. Improve exception messages to give more error context. Log every AssertionError as a backtrace. Use the 'raise Exception from e' syntax to modify the a previously raised exception 'e' into an exception with aditional context or different type. This also prevents the message that warns about newer exceptions being launch after an initial exception.
* utils: implement BIOS boot for windowsAlejandro Sirgo Rica2024-03-211-4/+10
| | | | | | | | | | | | | | | | | | Create ogboot.me and ogboot.secondboot as empty files and ogboot.firstboot with the value "iniciado" in the root of the BIOS Windows system partition. The files must contain data for GRUB to be able to write content, therefore these are created containing 3072 null bytes. The Windows boot process is handled by the "pxe" profile. There the files ogboot.me, ogboot.firstboot and ogboot.secondboot are used as a state machine to chose between booting Windows and ogLive. The first Windows boot happens if ogboot.me and ogboot.firstboot are identical, then "iniciado" is written in ogboot.firstboot. We skip this stage as we create ogboot.firstboot with 'iniciado'. The second Windows boot occurs if ogboot.me and ogboot.secondboot are boot identical, then "iniciado" is written in ogboot.secondboot. After the Windows boot ogLive is booted.
* utils: add disk index checks in get_partition_deviceAlejandro Sirgo Rica2024-03-211-1/+5
| | | | | Control the possibility of a bad disk index. Handle the checks in a similar fashion to the get_efi_partition function.
* utils: implement BIOS boot for LinuxAlejandro Sirgo Rica2024-03-212-1/+70
| | | | | | | | | Create a bios.py file to hold all the BIOS specific functions. Implement the _boot_bios_linux in Python. The new boot process tries to find the vmlinuz and initrd binaries at the desired partition. Then it tries to load them with kexec with the proper Grub boot params. One step closer to the removal of the boot legacy script.
* utils: handle exceptions caused by the hivex packageAlejandro Sirgo Rica2024-03-081-3/+6
| | | | | | The mage creation process was being interrupted by an error trying to read the Windows registry by the Hivex library. Now the exceptions are handled and an error is reported.