summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* utils: add BCD native supportAlejandro Sirgo Rica2024-07-291-0/+198
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implement update_bcd() as replacement of ogWindowsBootParameters. The legacy function modified the BCD of a freshly restored system invoking the privative binary "spartlnx". The script edits a set of BCD entries needed for a proper system boot. Each main BCD entry is identified by an UUID and contain a set of subnodes, these hold the configuration and entry information. Each node contains data in the form of key-value. Common BCD structure: 'Objects' ... The Boot Manager entry is always identified by the UUID 9dea862c-5cdd-4e70-acc1-f32b344d4795. Some entries always have the same UUID as identifier such the Boot Manager while other have different UUID depending on the system. To identify these entries with a not known UUID we query the value 'Type' of the node 'Description'. This contains a 32 bit value whose bytes codify the nature of the entry. We obtain 3 different values as a tuple, each value is the result of applying a bitmask to the Type value. These masks are 0xf0000000, 0x00f00000 and 0x000fffff. The resulting tuple of 3 values is then used to obtain the corresponding entry from a map. The data we modify from the BCD are disk and partition references to point to the new disk and partition in the system where the images are restored. Partition and disk information is stored as UUID in specific offsets inside binary data in the BCD. To update these we need to obtain the disk and partition UUIDs, then convert it to bytes as follows: original UUID: C4C61C51-3456-4733-96AD-AE760A41C392 UUID as bytes: 51 1C C6 C4 56 34 33 47 96 AD AE 76 0A 41 C3 92 The modified entries are: Resume from Hibernation, Windows Boot Loader OS, Windows Boot Loader Recovery, Windows Recovery, Boot Loader Settings, Windows Boot Manager and Windows Memory Diagnostic. Some of these options could be omited as the system restoration does not include a recovery partition so in this case all the recovery related entries just point to the main system partition. Most entries are edited modifying the value corresponding to the key 'Element' in 2 subnodes of the 'Elements' node. These subnodes are '11000001' and '21000001'. 'Objects' The 2 values stored in these 2 entries is binary data where we store the partition and disk bytes. We simply replace the byte representation of out UUIDS in the binary data. Partition is stored in offset 32 and the disk in offset 56. The exception is the Bootloader Recovery entry, in which the partition offset is 84 and the disk offset 108. Note that the legacy function only does a proper BCD edit in UEFI systems. The new implementation follows the same behavior with the possibility of implementing BCD modification under MBR partitions in the future. Set the field 16000009 (RECOVERY_ENABLED) to the value x00 to disable recovery in both Resume from Hibernation and Windows Boot Loader OS entries in the BCD. The system install does not include any recovery partition so it makes no sense to have it enabled. This commit is preparatory work for the new native postinstall code.
* utils: add win edit registry utilitiesAlejandro Sirgo Rica2024-07-291-0/+114
| | | | | | | | | | | | | | | | | Add winreg.py to the utils folder. Implement hive enum types, hive handler validation and validated get_* functions for nodes and registry values. Implement the utility hive operations through the Hivex library. This serves as preparatory work for BCD manipulation but it also has potential to improve registry usage in previous code. UCS-2 Little Endian is the prefered windows registry text encoding for binary content. Define a WINDOWS_HIVE_ENCODING global variable to use when encoding string to write in the win registry. This commit is preparatory work for the new native postinstall code.
* utils: add restore_windows_efi_bootloaderAlejandro Sirgo Rica2024-07-291-0/+31
| | | | | | | Add restore_windows_efi_bootloader to copy the EFI loader from the filesystem in the restored system into the EFI partition. This commit is preparatory work for the new native postinstall code.
* utils: add functions to obtain disk and partition UUIDsAlejandro Sirgo Rica2024-07-291-0/+25
| | | | | | | | Add get_partition_id and get_disk_id to obtain the UUID of a disk or partition as a string. These values are obtained invoking the program blkid as a subprocess. This commit is preparatory work for the new native postinstall code.
* utils: consolidate code to find efi loaderAlejandro Sirgo Rica2024-07-292-30/+29
| | | | | Add find_windows_efi_loader and find_linux_efi_loader to reduce code duplication and to centralize efi loader path modifications.
* utils: rename ogExtendFs to extend_filesystemAlejandro Sirgo Rica2024-07-292-2/+2
| | | | Use a more consistent name format for the ogExtendFs function.
* utils: fs: error out if mkfs failsOpenGnSys Support Team2024-07-292-7/+22
| | | | | | | | | | If formatting fails, log shows: mkfs.ntfs reports return code 1 for /dev/sda2 but ogclient reports success to the ogserver. Raise an exception so front-end gets an indication that formatting has failed.
* live: improve logging for file removal from cacheOpenGnSys Support Team2024-07-211-2/+8
| | | | Provide a bit more logging to make it easier to debug issues.
* live: add logging to indicate image file copy from sambaOpenGnSys Support Team2024-07-211-0/+2
| | | | For consistency with tiptorrent download.
* utils: consolidate logging to write checksum file to cacheOpenGnSys Support Team2024-07-211-2/+3
| | | | Move log message to function that is called both by unicast and tiptorrent.
* utils: missing f-string with filesystem resize errorv1.3.2-13OpenGnSys Support Team2024-07-151-1/+1
| | | | ... failed to resize {partdev} with ext4
* utils: clean up error reporting related to checksumOpenGnSys Support Team2024-07-151-2/+2
| | | | | | | | When checksum is not available, it displays: (2024-07-15 09:04:14) ogClient: [ERROR] - URL error when fetching checksum: Not Found "URL error" is leaking an internal implementation details, reword this report.
* live: add checksum field to image/create responseOpenGnSys Support Team2024-07-153-4/+17
| | | | Report image checksum to ogserver through HTTP response.
* live: check permissions when trying to fetch file via unicastv1.3.2-12OpenGnSys Support Team2024-06-271-0/+4
| | | | check sufficient permissions and bail out in case of issues.
* live: revisit error log when failing to validate checksumOpenGnSys Support Team2024-06-272-2/+5
| | | | | | | | Add explicit check for .full.sum after downloading it. Rewrite errors log, one of them is misleading when checksum validation fails, it refers to missing .full.sum, but it could be a different reason.
* live: add restricted execution mode to shell/runAlejandro Sirgo Rica2024-06-251-4/+19
| | | | | | | | | | | | | | | Try to find the script to run for a shell/run request in /opt/opengnsys/shell/, restricted mode is enabled if the script is found. Excute the script without shell=True and executable=OG_SHELL in restricted mode. Restricted mode is a safer execution method as it only executes code manually defined by the administrator. Each script needs to define a shebang, this way more than just bash is supported.
* rest: add cmd field to POST /shell/runOpenGnSys Support Team2024-06-214-4/+5
| | | | | echo command that has been run for storage in ogserver, until GET /shell/output is invoked.
* rest: add retcode field to POST /shell/runOpenGnSys Support Team2024-06-214-4/+5
| | | | | | | | provide return code as result to ogserver. Update virtual mode driver to return dummy value, although this command is unimplemented, this seems to be broken due to possible TypeError when accessing result from caller.
* live: handle no cache in tiptorrent and unicast restorev1.3.2-11Alejandro Sirgo Rica2024-06-071-1/+4
| | | | | | Properly report the lack of cache partition when restoring an image using TIPTORRENT and UNICAST as methods. Abort any restore in case of no cache partition.
* utils: remove bogus cache not found error after refresh commandAlejandro Sirgo Rica2024-06-071-1/+0
| | | | | | Remove log message "Cannot find device path to cache" spamming logs during operations such as refresh in clients without CACHE partition.
* utils: fix get_image_info regressionv1.3.2-10Alejandro Sirgo Rica2024-06-032-6/+6
| | | | | | | Fix image size, permissions and creation time. Improve error report related to these parameters now showing the exact cause of the problem if any occurred during the definition of image size, file permissions or image creation time values.
* live: add winrecov partition typeOpenGnSys Support Team2024-06-021-0/+1
| | | | | DE94BBA4-06D1-4D40-A16A-BFD50179D6AC is already set in in GUID_MAP as partition code, but not in GPT_PARTTYPES.
* live: revisit logging for checksum file creationv1.3.2-9OpenGnSys Support Team2024-05-311-4/+8
| | | | | Add logging before calculating checksum to provide a hint to user on what is going on while ogClient is still busy.
* src: cleanup ogGetImageInfoAlejandro Sirgo Rica2024-05-302-18/+12
| | | | | | Rename ogGetImageInfo to get_image_info. Move code from ogOperations.py to obtain image data into get_image_info.
* src: rename legacy.py into image.pyAlejandro Sirgo Rica2024-05-302-1/+1
| | | | | legacy.py contais mostly functions related to system images. Rename the file to better represent the contents in it.
* src: stop using hardcoded paths to cache image directoryAlejandro Sirgo Rica2024-05-302-9/+8
| | | | | | Use the constant OG_CACHE_IMAGE_PATH from cache.py to obtain the location of the directory where images are stored. This way the path can be changed from one single point.
* tiptorrent: check cache availability in tip_write_csumAlejandro Sirgo Rica2024-05-301-3/+4
| | | | | | Remove old TODO message. Check if the cache is available before trying to generate an image's checksum.
* src: add cache info to the image/restore responseAlejandro Sirgo Rica2024-05-303-6/+18
| | | | | | | | | | | | | | | | Add a 'cache' field into the json payload the client sends to the server after a restore operation so the server can update the new cache contents. Resquest response structure: { ... 'cache': [ {'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'}, {'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'} ] ... }
* virtual: handle copy error in image restoreAlejandro Sirgo Rica2024-05-301-3/+4
| | | | | Add a proper error report for the shutil.copy operation in image_restore() instead of silently returning.
* src: add POST cache/delete methodAlejandro Sirgo Rica2024-05-306-0/+74
| | | | | | | | | | | | | | | | | | | | Add API REST method to delete cache contents. Resquest payload structure: { 'images': ['windows.img', 'linux.img'] } The client will try to delete as many images in cache as available with names matching the list of filenames in the 'images' field. Resquest response structure: { 'cache': [ {'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'}, {'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'} ] }
* live: add cache contents to the /refresh payloadAlejandro Sirgo Rica2024-05-301-1/+41
| | | | | | | | | | | | | | | | | Add the list of images in the client's cache partition in the payload sent to the server. The information sent is a list of {image_name, img_size, checksum} elements where img_size is the size of the respective image in bytes. Resquest response structure: { ... 'cache': [ {'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'}, {'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'} ] ... }
* utils: cache: redefine cache path constantsAlejandro Sirgo Rica2024-05-271-8/+9
| | | | | | | | | Rename OGIMG as OG_IMAGE_PATH. Rename OGCACHE_MOUNTPOINT as OG_CACHE_PATH. Define OG_CACHE_IMAGE_PATH as OG_CACHE_PATH + OG_IMAGE_PATH. This will serve to have a unique point to obtain cache related paths.
* live: remove unused return value in image_restore()OpenGnSys Support Team2024-05-271-2/+1
| | | | Never used what configureOs() returns, remove it.
* live: incorrect reference to image checksum file in logsOpenGnSys Support Team2024-05-251-1/+1
| | | | checksum file name end by .img.full.sum, not .full.sum
* live: rename variable that stores json body in refresh()OpenGnSys Support Team2024-05-211-5/+5
| | | | Just a simple cleanup.
* live: parttypes: add EFI partition type in MBR partition schemeOpenGnSys Support Team2024-05-141-0/+1
| | | | Add 0xef partition type for EFI in MBR.
* live: partcodes: remove reference to websiteOpenGnSys Support Team2024-05-141-1/+0
| | | | Remove reference to external website in code.
* utils: rename cache_probe() to get_cache_dev_path()OpenGnSys Support Team2024-05-093-8/+11
| | | | | | | | This method reports the /dev path to cache partition, rename it. Add explicit check if blkid is successful. And add logging to report that device path to cache is not found.
* utils: add error checks to checksum file creation in tip_write_csumAlejandro Sirgo Rica2024-05-071-2/+5
| | | | | Add a check for potential permission or IO errors during the creation of the image checksum.
* utils: add mkdir error report in mount_mkdirAlejandro Sirgo Rica2024-05-071-1/+5
| | | | | | Add exception checks to the os.mkdir operation and log the error found. The previous implementation was too optimistic and only handled mount related errors.
* utils: remove unused json importAlejandro Sirgo Rica2024-05-071-1/+0
| | | | | The json library was a dependency during the development of the boot OS functions and it is no longer needed in that file.
* utils:fs: fix logging statement in get_filesystem_typeAlejandro Sirgo Rica2024-05-071-1/+1
| | | | | Call the error() logging function from the logging object instead of the non existent log variable previously referenced.
* utils:fs: add mkfs logs when return code is not 0Alejandro Sirgo Rica2024-05-071-12/+24
| | | | | | Report mkfs failure for every partition. This does not raise an exception as that would skip partprobe operations and the mkfs operations in the next potentially well formated partitions.
* live: report LINUX-SWAP instead of SWAPv1.3.2-8OpenGnSys Support Team2024-05-071-0/+3
| | | | | | | ogCP expects LINUX-SWAP to specify a swap filesystem. Add a similar workaround to the one that is done for VFAT for symmetry between inputs and outputs that circulate over the API.
* live: rewrite log in case tiptorrent client failsOpenGnSys Support Team2024-05-061-1/+1
| | | | | Specify that image file cannot be found in cache because tiptorrent has failed, otherwise it is confusing.
* live: restore partprobe before building filesystemOpenGnSys Support Team2024-05-061-1/+4
| | | | Otherwise mkfs silently fails because OS reports out-of-sync partition table.
* live: force flush to disk after partition table is writtenOpenGnSys Support Team2024-05-061-0/+1
|
* live: partprobe breaks with mounted partitionsOpenGnSys Support Team2024-05-061-2/+11
| | | | | | | | | | partprobe requires that all disk partitions are unmounted. partprobe needs to be called to report the OS that the partition table has changed, otherwise ogclient reports incorrect partition information. iterate over the partition list and mount cache after partprobe is called.
* live: umount all partitions before partition setupOpenGnSys Support Team2024-05-061-2/+3
| | | | | If new partition layout is specified, unmount cache and any other partition under /mnt.
* live: fix omited error report in tip_client_getAlejandro Sirgo Rica2024-05-061-7/+6
| | | | | | | tip_client_get raises the proper error exceptions but the except block in _restore_image_tiptorrent overwrites the reported error. Move the raise statements in _restore_image_tiptorrent outside of the except block.