qortal

Commit Graph

Author	SHA1	Message	Date
CalDescent	9c20967d24	Catch NPE seen a couple of times in Systray.setTrayIcon()	3 years ago
CalDescent	79bbadad2f	Initial implementation of arbitrary data build queue This adds the loadAsynchronously() method to ArbitraryDataReader, in addition to the existing loadSynchronously() method. When requesting a website in a browser, previously the building of the resource's layers would be done synchronously in the API handler. This understandably caused many issues, so the building is now done asynchronously by a dedicated thread. A loading screen is shown in its place which auto refreshes every second until the build has completed.	3 years ago
CalDescent	c3b44cee94	Fixed bugs when computing directory digest. The order was being lost when adding to the second List, which was producing different hashes on different systems. It also doesn't make sense to convert paths to lowercase, given that we want our validation to be case sensitive.	3 years ago
CalDescent	1968496ce1	Invalidate the cache if hash validation fails, so that it can be rebuilt next time.	3 years ago
CalDescent	e1feb46de9	Put test data in a separate folder to real data.	3 years ago
CalDescent	f51a082049	Validate the previous state's hash each time a new layer is applied. It's possible that this concept will struggle in the real world if operating systems, virus scanners, etc start interfering with our file stucture. Right now it is using a zero tolerance approach when checking the validity of each layer. We may choose to loosen this slightly if we encounter problems, e.g. by excluding hidden files. But for now it is best to be as strict as possible.	3 years ago
CalDescent	0f1927b4b1	Take a hash of the previous state's directory structure and file contents, and then include that hash in the patch metadata (when creating a new patch). This allows the integrity of the layers to be validated as each one is applied.	3 years ago
CalDescent	9baccc0784	Improved HTTP response generation when serving websites.	3 years ago
CalDescent	95c9cc7f99	Added ArbitraryDataCache This decides whether to build a new state or use an existing cached state when serving a data resource. It will cache a built resource until a new transaction (i.e. layer) arrives. This drastically reduces load, and still allows for almost instant propagation of new layers.	3 years ago
CalDescent	0ed8e04233	Arbitrary data metadata classes moved to their own package.	3 years ago
CalDescent	b46c328811	Fixed bug in FilesystemUtils.copyAndReplaceDirectory()	3 years ago
CalDescent	c7d88ed95b	Added "cache" file to the .qortal metadata folder. This is used to store the transaction signature and build timestamp for each built data resource. It involved a refactor of the ArbitraryDataMetadata class to introduce a subclass for each file ("patch" and "cache"). This allows more files to be easily added later.	3 years ago
CalDescent	94da1a30dc	When merging two states, validate that the transaction signature we are using for the "before" layer matches the "previous transaction signature" that is baked into the "after" layer. This defends against a missing or out-of-order transaction. If this ever fails validation, we may need to rethink the way we are requesting transactions. But in theory this shouldn't happen, given that the "last reference" field of a transaction ensures that out-of-order transactions are invalid already.	3 years ago
CalDescent	219a5db60c	Fixed circular bug in ArbitraryDataMetadata.getPreviousSignature()	3 years ago
CalDescent	a5cfedcae9	Exclude the .qortal metadata directory in all diffs. Also improved logging.	3 years ago
CalDescent	9b8a632f37	Include the .qortal folder in the merge output, since it will be needed for validation. We may choose to exclude it from the final output path, but for now it is left there to make debugging easier.	3 years ago
CalDescent	be0426d9a2	Improved logging	3 years ago
CalDescent	8fac0a02e5	Use the /.qortal/patch file to apply differences when merging together two layers, rather than walking through the file tree like we did before.	3 years ago
CalDescent	fa696a2901	Log exceptions when publishing data updates.	3 years ago
CalDescent	f5615b1c54	Don't exclude hidden files in the zips, as the .qortal folder needs to be included.	3 years ago
CalDescent	9850c294d1	Fixed various issues relating to syncing data for transactions without any chunks.	3 years ago
CalDescent	8929f32068	No longer creating the ".removed" files.	3 years ago
CalDescent	3019bb5c97	Enforce minBlockchainPeers in ArbitraryDataManager, as there is no point in trying to request data files when we don't have the minimum number of peers.	3 years ago
CalDescent	95044d27ce	Fixed NPE caused by having an arbitrary transaction with no chunks (which is expected if the total data size is less than the chunk size).	3 years ago
CalDescent	16ac92b2ef	Write patch metadata to a file inside a hidden ".qortal" folder which is included with each patch. This can be used in place of the existing ".removed" placeholder files to track removals.	3 years ago
CalDescent	f095964f7b	Fixed edge case.	3 years ago
CalDescent	cfba793fcf	Fixed bugs in ArbitraryDataFile, introduced when switching to the new temporary path.	3 years ago
CalDescent	c50a11e58a	Cleanup intermediate paths in ArbitraryDataWriter.	3 years ago
CalDescent	a7282a5794	Renamed encrypted and zipped files.	3 years ago
CalDescent	efaf313422	When creating an ArbitraryDataFile from a path, make sure to move the file to the correct location within the data directory. This bug was introduced now that the temp directory is contained within the data directory. Without this, it would leave it in the temp folder and then fail at a later stage.	3 years ago
CalDescent	d9f5753f58	Ensure parent directories exist when creating a diff.	3 years ago
CalDescent	179318f1d8	Ensure a patch exists after creating it.	3 years ago
CalDescent	dd33d24346	Fixed bug with identifier in ArbitraryDataWriter	3 years ago
CalDescent	fa684eabab	Removed ArbitraryDataPatches class This was essentially just a wrapper for a single method.	3 years ago
CalDescent	4b5bd6eed7	Cleanup temporary files used when building data directories. Also added a safety net to only delete files that are without our data or temp directories.	3 years ago
CalDescent	63f5946527	Switched all system-generated temp paths to a user specified "tempDataPath". This ensures that the temporary files are being kept with the rest of the data, rather than somewhere inappropriate such as on flash storage. It also allows the user to locate them somewhere else, such as on a dedicated drive.	3 years ago
CalDescent	8dac3ebf96	More miscellaneous refactoring	3 years ago
CalDescent	09e783fbf6	Renamed a few variables and methods that weren't caught in previous refactors.	3 years ago
CalDescent	b18b686545	Renamed DATA_FILE to ARBITRARY_DATA_FILE and GET_DATA_FILE to GET_ARBITRARY_DATA_FILE. This will break existing data nodes, but this is okay, as nothing is in production yet.	3 years ago
CalDescent	e99ea41117	Renamed DataFileMessage to ArbitraryDataFileMessage and GetDataFileMessage to GetArbitraryDataFileMessage	3 years ago
CalDescent	172a37ec8c	Renamed org.qortal.storage package to org.qortal.arbitrary	3 years ago
CalDescent	da6b341b63	Renamed DataFile to ArbitraryDataFile, and DataFileChunk to ArbitraryDataFileChunk	3 years ago
CalDescent	16d93b1775	Renamed newly added classes to ArbitraryData.java instead of DataFile.java	3 years ago
CalDescent	e15cf063c6	Initial implementation of data patches/updates This adds support for the PATCH method in addition to the existing PUT method. Currently, a patch includes only files that have been added or modified, as well as placeholder files to indicate those that have been removed. This is not production ready, as I am hoping to create patches on a more granular level - i.e. just the modified bytes of each file. It would also make sense to track deletions using a metadata/manifest file in a hidden folder. It also adds early support of accessing files using a name rather than a signature or hash.	3 years ago
CalDescent	5ac9e3e47a	Fixed recently introduced bugs in arbitrary transaction transformation.	3 years ago
CalDescent	743a61bf49	Fixed bug which overwrote the data file, causing it to be invalid.	3 years ago
CalDescent	c790ea07dd	Started abstracting the file processing code away from the API handlers, and making it more modular.	3 years ago
CalDescent	e259a09b89	Fixed merge issues.	3 years ago
CalDescent	6472d8438a	Merge remote-tracking branch 'qortal/master' # Conflicts: # pom.xml # src/main/java/org/qortal/api/ApiError.java # src/main/java/org/qortal/settings/Settings.java # src/main/resources/i18n/ApiError_en.properties	3 years ago
CalDescent	dc8a402a4a	Revert "Removed all cross-chain code. It's not needed for data nodes." This reverts commit `cc59510cd0`. # Conflicts: # src/main/java/org/qortal/controller/Controller.java	3 years ago

1 2 3 4 5 ...

1415 Commits (9c20967d248c4bcacf57cfdb0b2efe6ab744d4b2) All Branches Search

1415 Commits (9c20967d248c4bcacf57cfdb0b2efe6ab744d4b2)

All Branches