[HPCC-23383] Rudimentary S3 and Azure Blob reading - HPCC

XML

Word

Printable

Rudimentary S3 and Azure Blob reading

relates to

HPCC-23412 Reading/writing logical files to S3/azure

HPCC-23427 Search for situations where IFiles are created, destroyed and then created

HPCC-25249 Review all disk buffer sizes

HPCC-24091 Archive plugin cores if opening zip file with no subfile

HPCC-23188 Investigate supporting Azure blobs

HPCC-23375 More complete storage plane solution

(1 relates to)

Progress

1.	Implement S3 IFile hook	Accepted	Gavin Halliday
2.	Avoid creating a directory unless a file create fails.	Accepted	Gavin Halliday
3.	~CompressedFile performs two writes	Accepted	Gavin Halliday
4.	Do not rename outputs files when writing to blobs	New	Unassigned
5.	Check containers are created if they do not exists	Accepted	Gavin Halliday
6.	check uses of getRemotePath() and do not prefix with //ip/	Accepted	Gavin Halliday
7.	Use asynchronous writes for output	Accepted	Gavin Halliday
8.	Think through implications of blobs for buffering	New	Unassigned
9.	Add asynchronous read ahead for blobs	Accepted	Gavin Halliday
10.	Ensure credentials are not leaked from filenames	Accepted	Gavin Halliday
11.	Cache the client object for a storage account	Accepted	Gavin Halliday
12.	Protect azure blob writing larger buffer sizes	New	Unassigned