qortal

Commit Graph

Author	SHA1	Message	Date
CalDescent	b2c0915a71	Removed accidentally duplicated code This was causing two instances of the build manager to run.	3 years ago
CalDescent	095083bcfb	Use lowercase directory names for consistency	3 years ago
CalDescent	4ba72f7eeb	Regularly clean up old and unused files/folders in the temp directory Also added code to purge built resource caches, but it is currently disabled. This will become more useful once we implement local storage limits.	3 years ago
CalDescent	6cb39795a9	Removed requirement to have connected peers in order to cleanup directories.	3 years ago
CalDescent	00ba16f536	Fixed incorrect ArbitraryTransactionTransformer layout.	3 years ago
CalDescent	988a839623	Improved response value of ArbitraryDataFile.deleteAllChunks() as it was inaccurate.	3 years ago
CalDescent	8fa61e628c	Delete files related to transactions that have a more recent PUT A PUT creates a new base layer meaning anything before that point is no longer needed. These files are now deleted automatically by the cleanup manager. This involved relocating a lot of the cleanup manager methods into a shared utility, so that they could be used by the arbitrary data manager. Without this, they would be fetched from the network again as soon as they were deleted.	3 years ago
CalDescent	8f3620e07b	Fixed bug introduced in commit `51b1256`	3 years ago
CalDescent	190f70f332	Removed unused, buggy code in HSQLDBArbitraryRepository.save() It's safer to throw an exception and point the user towards ArbitraryDataWriter, rather than maintaining unused code.	3 years ago
CalDescent	6730683919	Added arbitrary data cleanup manager This deletes redundant copies of data, and also converts complete files to chunks where needed. The idea being that nodes only hold chunks, since they currently are much more likely to serve a chunk to another peer than they are to serve a complete file. It doesn't yet cleanup files that are unassociated with transactions, nor does it delete anything from the _temp folder.	3 years ago
CalDescent	51b12567e8	Switched ArbitraryDataFile.filePath from String to Path	3 years ago
CalDescent	d01cdeded8	Added pagination to ArbitraryDataManager This improves scalability but isn't sufficient for a long term solution. TODO: It probably makes sense to add an additional query for recent transactions only, so that they are fetched quickly.	3 years ago
CalDescent	d22a03f1a5	Fixed misleading exception string.	3 years ago
CalDescent	ab0aeec434	Default method of serving websites switched from signature to name. Before: GET /site/:signature GET /site/name/:name After: GET /site/signature/:signature GET /site/:name	3 years ago
CalDescent	47ff51ce4e	Use a random last reference on the very first transaction for an account This is needed because we want to allow brand new accounts to publish data without a fee. A similar approach to CrossChainResource.buildAtMessage(). We already require PoW on all arbitrary transactions, so no additional logic beyond this should be needed.	3 years ago
CalDescent	1d62ef357d	Check for null last references at the beginning of the process when creating an arbitrary tx.	3 years ago
CalDescent	59ef66f46d	Handle shutdowns when zipping a large number of files.	3 years ago
CalDescent	77479215a6	Throw an exception if a patch file doesn't exist.	3 years ago
CalDescent	11da1f72b1	Added convenience method to make the code more readable.	3 years ago
CalDescent	6375b9d14d	Fixed bug in getLatestTransaction() filtering.	3 years ago
CalDescent	42bc12f56d	Call cleanupQueues() from the Controller	3 years ago
CalDescent	8515112811	Fixed unused variable issue	3 years ago
CalDescent	bedb87674b	FAILURE_TIMEOUT set to 5 minutes	3 years ago
CalDescent	029c038a49	Catch runtime exceptions (e.g. IllegalStateException) when using the arbitrary data reader/writer.	3 years ago
CalDescent	95e905a5ae	If a build fails, prevent any rebuilds for 5 minutes. This prevents a resource with build problems from getting into a loop due to the browser requesting a rebuild as soon as it fails.	3 years ago
CalDescent	8a8ec32f2c	Added java-diff-utils to pom.xml This was accidentally missing from commit `cb6fc46`.	3 years ago
CalDescent	cd958398af	Exclude all data* paths from gitignore.	3 years ago
CalDescent	2eedafd506	Log the error if a build fails.	3 years ago
CalDescent	9a88c0d579	Apply the unified-diff patch when combining layers.	3 years ago
CalDescent	cb6fc466d1	Create a "unified-diff" patch using java-diff-utils instead of including the entire modified file. I still need to test how well this works with binary files.	3 years ago
CalDescent	a6154cbb43	Don't allow a new layer to be created if it matches the existing state.	3 years ago
CalDescent	9c20967d24	Catch NPE seen a couple of times in Systray.setTrayIcon()	3 years ago
CalDescent	79bbadad2f	Initial implementation of arbitrary data build queue This adds the loadAsynchronously() method to ArbitraryDataReader, in addition to the existing loadSynchronously() method. When requesting a website in a browser, previously the building of the resource's layers would be done synchronously in the API handler. This understandably caused many issues, so the building is now done asynchronously by a dedicated thread. A loading screen is shown in its place which auto refreshes every second until the build has completed.	3 years ago
CalDescent	c3b44cee94	Fixed bugs when computing directory digest. The order was being lost when adding to the second List, which was producing different hashes on different systems. It also doesn't make sense to convert paths to lowercase, given that we want our validation to be case sensitive.	3 years ago
CalDescent	1968496ce1	Invalidate the cache if hash validation fails, so that it can be rebuilt next time.	3 years ago
CalDescent	e1feb46de9	Put test data in a separate folder to real data.	3 years ago
CalDescent	f51a082049	Validate the previous state's hash each time a new layer is applied. It's possible that this concept will struggle in the real world if operating systems, virus scanners, etc start interfering with our file stucture. Right now it is using a zero tolerance approach when checking the validity of each layer. We may choose to loosen this slightly if we encounter problems, e.g. by excluding hidden files. But for now it is best to be as strict as possible.	3 years ago
CalDescent	0f1927b4b1	Take a hash of the previous state's directory structure and file contents, and then include that hash in the patch metadata (when creating a new patch). This allows the integrity of the layers to be validated as each one is applied.	3 years ago
CalDescent	9baccc0784	Improved HTTP response generation when serving websites.	3 years ago
CalDescent	95c9cc7f99	Added ArbitraryDataCache This decides whether to build a new state or use an existing cached state when serving a data resource. It will cache a built resource until a new transaction (i.e. layer) arrives. This drastically reduces load, and still allows for almost instant propagation of new layers.	3 years ago
CalDescent	0ed8e04233	Arbitrary data metadata classes moved to their own package.	3 years ago
CalDescent	b46c328811	Fixed bug in FilesystemUtils.copyAndReplaceDirectory()	3 years ago
CalDescent	c7d88ed95b	Added "cache" file to the .qortal metadata folder. This is used to store the transaction signature and build timestamp for each built data resource. It involved a refactor of the ArbitraryDataMetadata class to introduce a subclass for each file ("patch" and "cache"). This allows more files to be easily added later.	3 years ago
CalDescent	94da1a30dc	When merging two states, validate that the transaction signature we are using for the "before" layer matches the "previous transaction signature" that is baked into the "after" layer. This defends against a missing or out-of-order transaction. If this ever fails validation, we may need to rethink the way we are requesting transactions. But in theory this shouldn't happen, given that the "last reference" field of a transaction ensures that out-of-order transactions are invalid already.	3 years ago
CalDescent	219a5db60c	Fixed circular bug in ArbitraryDataMetadata.getPreviousSignature()	3 years ago
CalDescent	a5cfedcae9	Exclude the .qortal metadata directory in all diffs. Also improved logging.	3 years ago
CalDescent	9b8a632f37	Include the .qortal folder in the merge output, since it will be needed for validation. We may choose to exclude it from the final output path, but for now it is left there to make debugging easier.	3 years ago
CalDescent	be0426d9a2	Improved logging	3 years ago
CalDescent	8fac0a02e5	Use the /.qortal/patch file to apply differences when merging together two layers, rather than walking through the file tree like we did before.	3 years ago
CalDescent	fa696a2901	Log exceptions when publishing data updates.	3 years ago

1 2 3 4 5 ...

1446 Commits (b2c0915a712e4b5e21543b5634ad14e171112c31) All Branches Search

1446 Commits (b2c0915a712e4b5e21543b5634ad14e171112c31)

All Branches