summaryrefslogtreecommitdiffstats
path: root/src/utils/fs.py
Commit message (Collapse)AuthorAgeFilesLines
* src: update license headerOpenGnSys Support Team2024-11-281-1/+1
|
* src: consolidate compute_md5 functionsAlejandro Sirgo Rica2024-10-021-0/+17
| | | | | | | | | | Add compute_md5 function in src/utils/fs.py Remove identical md5 functions from src/live/ogOperations.py and src/utils/tiptorrent.py Move error checks from ogOperations.py into compute_md5 function in src/utils/fs.py
* src: check if the system is hibernated before /image/createAlejandro Sirgo Rica2024-10-021-4/+7
| | | | | | | Mount the system partition in readonly mode and check for the hiberfil.sys file if the target system is a Windows. Fail the image creation process if the target system is hibernated.
* src: report used and free partition data in bytesAlejandro Sirgo Rica2024-09-251-11/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | Add "used_size" and "free_size" to the partition data and the cache data. Old response from ogClient for /cache/delete, /cache/fetch and /image/restore: { 'cache': [ {'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'}, {'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'} ] } New response: { 'cache': { 'used_size': 4520232322423, 'free_size': 48273465287452945, 'images': [ {'name': 'windows.img', 'size': 2432370213, checksum: '5d4dcc677bc19f40a647d0002f4ade90'}, {'name': 'linux.img', 'size': 243234534213, checksum: '3eb22f888f88a55ad954f55644e1192e'} ] } }
* utils: rename ogExtendFs to extend_filesystemAlejandro Sirgo Rica2024-07-291-1/+1
| | | | Use a more consistent name format for the ogExtendFs function.
* utils: fs: error out if mkfs failsOpenGnSys Support Team2024-07-291-5/+17
| | | | | | | | | | If formatting fails, log shows: mkfs.ntfs reports return code 1 for /dev/sda2 but ogclient reports success to the ogserver. Raise an exception so front-end gets an indication that formatting has failed.
* utils: missing f-string with filesystem resize errorv1.3.2-13OpenGnSys Support Team2024-07-151-1/+1
| | | | ... failed to resize {partdev} with ext4
* utils: add mkdir error report in mount_mkdirAlejandro Sirgo Rica2024-05-071-1/+5
| | | | | | Add exception checks to the os.mkdir operation and log the error found. The previous implementation was too optimistic and only handled mount related errors.
* utils:fs: fix logging statement in get_filesystem_typeAlejandro Sirgo Rica2024-05-071-1/+1
| | | | | Call the error() logging function from the logging object instead of the non existent log variable previously referenced.
* utils:fs: add mkfs logs when return code is not 0Alejandro Sirgo Rica2024-05-071-12/+24
| | | | | | Report mkfs failure for every partition. This does not raise an exception as that would skip partprobe operations and the mkfs operations in the next potentially well formated partitions.
* utils: fs: incorrect indentationOpenGnSys Support Team2024-04-221-1/+1
|
* utils: fs: skip shrink/extend operation for fatOpenGnSys Support Team2024-04-191-0/+4
| | | | fat does not support this operation, skip it
* utils: fs: remove unused variable in _reduce_resize2fsv1.3.2-7OpenGnSys Support Team2024-04-191-1/+0
| | | | ret is set but never used in this method
* utils: fs: report error when failing to growOpenGnSys Support Team2024-04-191-2/+2
| | | | not really a warning, report an error instead
* utils: fs: set return value in ogReduceFsOpenGnSys Support Team2024-04-191-2/+3
| | | | | | | | otherwise error path uses uninitialized variable File "/opt/opengnsys/ogClient/src/utils/fs.py", line 112, in ogReduceFs return ret UnboundLocalError: local variable 'ret' referenced before assignment
* src: log backtrace in unhandled error casesAlejandro Sirgo Rica2024-04-031-6/+7
| | | | | | | | | | | | | | | | | | | Log an error message in known error cases and log a backtrace otherwise. Define a new error type OgError to be used in all the 'raise' blocks to define the error message to log. The exception propagates until it reaches send_internal_server_error() where the exception type is checked. If the type is OgError we log the exception message. Logs the backtrace for other types. The initial error implementation printed a backtrace everytime an error ocurred. The next iteration changed it to only print a backtrace in a very particular case but ended up omiting too much information such as syntax errors or unknown error context. The actual implementation only logs the cases we already cover in the codebase and logs a bracktrace in the others, enabling a better debugging experience.
* utils: remove redundant return statements from mount_mkdirAlejandro Sirgo Rica2024-03-261-3/+1
| | | | | | Remove return statement the program won't reach and simplify the return logic with a return from an if and a fallback return statement.
* src: centralize error logging into send_internal_server_errorAlejandro Sirgo Rica2024-03-211-8/+4
| | | | | | | | | | | | | | | | | | | | Use only the exception messages as the main resource for error messages. The previous error code had string duplication in the form of: logging.error('msg here') raise Exception('msg here') That approach also has the downside of having log duplication as it had the local logging.err() and a global logging.exception() inside send_internal_server_error capturing the exception message. The actual code only requires raising an exception with a proper error message. Improve exception messages to give more error context. Log every AssertionError as a backtrace. Use the 'raise Exception from e' syntax to modify the a previously raised exception 'e' into an exception with aditional context or different type. This also prevents the message that warns about newer exceptions being launch after an initial exception.
* fs: add swap support and improve mkfs loggingOpenGnSys Support Team2024-02-221-3/+15
|
* fs: improve value parsing from command outputAlejandro Sirgo Rica2024-02-191-7/+29
| | | | | | value extraction did not have error checking and was handled in a one-liner. The actual implementation expands the parsing logic and moves it into a function.
* fs: return unknown if blkid failsOpenGnSys Support Team2024-02-151-1/+7
| | | | instead of rising an exception
* fs: logging ext4 resize errorOpenGnSys Support Team2024-02-151-2/+4
| | | | log error in case resize2fs fails.
* fs: check if ntfsresize actually succeded to shrink filesystemOpenGnSys Support Team2024-02-151-1/+4
| | | | | According to ntfsresize.c, this retuns 0 in case nothing needs to be done. It should be safe to check for non-zero error and bail out in that case.
* fs: disentagle dry-run ntfsresize loop to probe for best shrink sizeOpenGnSys Support Team2024-02-151-8/+16
| | | | | | | | Revisit 5056b8f0d5ab ("fs: validate ntfsresize dry-run output") that has introduced a possible infinity loop. Disentangle this loop while at it: iterate until best smallest size is found by probing.
* src: improve error check in image_create and image_restoreAlejandro Sirgo Rica2024-02-141-3/+17
| | | | | | | | | cover more error cases where exceptions need to be raised. check return code in the invoked subprocess. restoreImageCustom has been intentionally left behind, it is unclear what this custom script returns on success and error.
* fs: validate ntfsresize dry-run outputv1.3.2-5OpenGnSys Support Team2024-01-111-2/+3
| | | | | | | | | | | | | | | | validate 'Needed relocations: ' is in place before stepping on the split chunks (2024-01-11 10:28:16) ogClient: [ERROR] - Exception when running "image create" subprocess Traceback (most recent call last): File "/opt/opengnsys/ogClient/src/live/ogOperations.py", line 454, in image_create ogReduceFs(disk, partition) File "/opt/opengnsys/ogClient/src/utils/fs.py", line 105, in ogReduceFs _reduce_ntfsresize(partdev) File "/opt/opengnsys/ogClient/src/utils/fs.py", line 235, in _reduce_ntfsresize extra_size = int(out_resize_dryrun.split('Needed relocations : ')[1].split(' ')[0])*1.1+1024 IndexError: list index out of range if not present, no need to adjust size
* fs: fix subprocess input inside _extend_resize2fsJose M. Guisado2023-06-131-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | The subprocess module expects bytes-like object for "input" parameter by default. Passing a string object result in the following error: (2023-06-13 14:44:43) ogClient: [ERROR] - Exception when running "image create" subprocess (2023-06-13 14:44:43) ogClient: [ERROR] - Unexpected error Traceback (most recent call last): File "/opt/opengnsys/ogClient/src/live/ogOperations.py", line 465, in image_create ogExtendFs(disk, partition) File "/opt/opengnsys/ogClient/src/utils/fs.py", line 124, in ogExtendFs _extend_ntfsresize(partdev) File "/opt/opengnsys/ogClient/src/utils/fs.py", line 250, in _extend_ntfsresize proc = subprocess.run(cmd, input='y') File "/usr/lib/python3.8/subprocess.py", line 495, in run stdout, stderr = process.communicate(input, timeout=timeout) File "/usr/lib/python3.8/subprocess.py", line 1013, in communicate self._stdin_write(input) File "/usr/lib/python3.8/subprocess.py", line 962, in _stdin_write self.stdin.write(input) TypeError: a bytes-like object is required, not 'str' Fixes: dd999bfe34e7 ("utils: rewrite ogReduceFs")
* fs: fix bug when ntfsresize reports nothing to doJose M. Guisado2023-06-131-0/+3
| | | | | | | | There is a corner case in which a target NTFS filesystem is already shrunken. When this happens ntfsresize text output parsing breaks. Check when ntfsresize reports nothing to do, warn the user about this and stop the dry-run ntfsresize loop.
* fs: fix typo inside _extend_resize2fsJose M. Guisado2023-06-131-4/+4
| | | | | | | | | | _extend_ntfsresize contains an incorrect variable name inside subprocess.run referring the resize command value. Simplify this variable name inside each specific _extend_* function: s/cmd_resize2fs/cmd s/cmd_ntfsresize/cmd
* utils: add umount_all functionJose M. Guisado2023-05-091-0/+9
| | | | | | | | Add utility function to unmount any mountpoint present in the /mnt folder. This function is a simplified version of the legacy bash function ogUnmountAll used in several operations.
* utils: rewrite ogExtendFsJose M. Guisado2023-05-091-7/+28
| | | | | | | | | | | | | | | | | | | | Drop subprocess call to bash function ogExtendFs. Use a native python solution with subprocess calls to the required underlying tools. Use get_filesystem_type to get the present filesystem from a partition and call the corresponding filesystem grow function. Filesystem specific functions are declared "_extend_{filesystem}" and should not be imported elsewhere. Each filesystem specific function wraps a subprocess call to the required underlying program: - NTFS filesystems: "ntfsresize -f [partition]" - ext4 filesystems: "resize2fs -f [partition]" Set NTFS related subprocess stdin to 'y' because human input cannot be unset with other ntfsresize parameters.
* utils: rewrite ogReduceFsJose M. Guisado2023-05-021-9/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | Drop subprocess call to bash function ogReduceFs. Use a native python solution with subprocess calls to the required underlying tools. Use get_filesystem_type to get the filesystem from a partition and call the corresponding supported filesystem shrink function. Filesystem specific functions are declared "_reduce_{filesystem}" and should not be imported elsewhere. In case of NTFS filesystems, the output of 'ntfsresize' is processed directly. This is dirty, but we can expect no changes to the output strings if we read the following comment in the nftsresize.c source code: https://github.com/tuxera/ntfs-3g/blob/edge/ntfsprogs/ntfsresize.c#L12 ntfsresize requires to do previous dry-run executions to confirm that the resizing is possible. If a dry-run fails but a 10% increase in size is still smaller than original filesystem then retry the operation until dry-run reports sucess or the size increase is bigger than original. If resizing to a smaller ntfs filesystem is not possible then ogReduceFs will do nothing.
* utils: add get_filesystem_type functionJose M. Guisado2023-04-251-0/+12
| | | | | Retrieve filesystem type from a partition using get_filesystem_type. Encapsulates a subprocess call to blkid.
* utils: add fs.pyJose M. Guisado2023-02-081-1/+66
| | | | | | | | | | | Adds utility module which wraps several mkfs.* calls as a subprocess. The main utility function is mkfs(fs, disk, partition, label), which subsequently calls the corresponding mkfs_*(partition_device) function. mkfs() supports specifying a drive label where supported. Other modules using fs.py should call mkfs() only.
* utils: minor fix for ogReduceFs and ogExtendFsJose M. Guisado2022-09-141-10/+10
| | | | | | | | | | Do not return the subprocess result for ogReduceFs/ogExtendFs. ogReduceFs works with or without the target filesystem mounted. ogExtendFs requires the target filesystem to be mounted. 'ogMount' legacy script invocation should be replaced by a better mount/umount wrapper.
* image_create: partial integration into pythonv1.2.2Jose M. Guisado2022-08-241-1/+27
| | | | | | | | | | | Integrates some parts of this operation into native code, eg: the md5 checksum computation. Wraps non native processes and commands using the subprocess module. For example, legacy.py stores bash commands pending integration. Supports python >=3.6, expected until more modern ogLives are put into production environments.
* utils: mount_mkdir success if target is a mountpointJose M. Guisado2022-06-011-0/+6
| | | | | | | | Returns true if target is already a mountpoint. Does not call mount. It's possible that another device might be mounted in the target mountpoint. A future check between the source and target for equal device major:minor must be added.
* probe: detect 64 bit operating systemsJose M. Guisado2022-05-091-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | OpenGnsys partition images store OS information, including the OS name appended with "64 bits" when the OS is meant for 64 bit machines. The detected OS name when probing (refresh) is important, if it differs from what's stored in the DB OpenGnsys will wipe last image restored information when running a refresh. See actualizaConfiguracion from legacy ogserver (ogAdmServer.c) code: dato = dbi_result_get_uint(result, "idnombreso"); if (idsoi == dato) { swu = false; } ... if (swu) { result_update = dbi_conn_queryf(dbi->conn, "UPDATE ordenadores_particiones SET " " codpar=0x%s," " tamano=%s," " uso=%s," " idsistemafichero=%d," " idnombreso=%d," " idimagen=0," " idperfilsoft=0," " fechadespliegue=NULL" " WHERE idordenador=%d AND numdisk=%s AND numpar=%s",
* Add utils modulesJose M. Guisado2022-04-211-0/+60
* disk.py Disk discovery * fs.py Uses psutil to fetch fs usage information * menu.py ogBrowser menu generation * net.py: gets nic status information IP address, MAC address and ethernet speed. * probe.py: probes mountpoints for operating systems Uses hivexget command to try fetching Windows installation information. Looks for /etc/os-release for probing linux systems.