Linux Lite Forums

Hardware - Support => Other => Topic started by: m654321 on January 12, 2018, 08:03:43 AM

Title: How do I combine several small HDDs to make one large HDD?
Post by: m654321 on January 12, 2018, 08:03:43 AM
Over the years, I've collected several laptop HDDs, particularly now that we've converted all our household's computers to the much faster SSDs.
However, I don't really want to get rid of these old drives, especially if I can find a use for them.

If I could somehow combine these HDDs (they vary in size from 250GB to 1TB) into one large drive, then I could use them
for external data storage, via USB connectivity.

Could someone either explain to me how I can do this, or at least point me in the right direction?

Many thanks - much appreciated.
Mike
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: bitsnpcs on January 12, 2018, 09:21:39 AM
Hello Mike,

for Linux - https://unix.stackexchange.com/questions/329790/how-to-merge-multiple-hard-drives (https://unix.stackexchange.com/questions/329790/how-to-merge-multiple-hard-drives)

Additional info click through on the above reply - https://www.howtoforge.com/linux_lvm


If you also wish to use it on windows machines

https://www.youtube.com/watch?v=PCQu-WmE4SE (https://www.youtube.com/watch?v=PCQu-WmE4SE)
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: Scott on January 12, 2018, 10:22:18 AM
@bitsnpcs - Very good information, thank you.
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: bitsnpcs on January 12, 2018, 11:51:29 AM
@Scott I'll try it too when I replace the 32 bit desktops this year.



Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: Moltke on January 12, 2018, 07:09:46 PM
@bitsnpcs  Such a nice informaton! I didn't even ever think this was possible. Bookmarked!
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: bitsnpcs on January 12, 2018, 09:05:12 PM
@Moltke if many spare hdd combined -

https://www.youtube.com/watch?v=d3ufUXZafAk (https://www.youtube.com/watch?v=d3ufUXZafAk)

https://www.youtube.com/watch?v=tUL2vesXu0o (https://www.youtube.com/watch?v=tUL2vesXu0o)


It seems there can be made "pools", a combined group effort, it could be done by a community, as a source of funding for a project/s.
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: m654321 on January 13, 2018, 02:59:46 PM
Hello Mike,

for Linux - https://unix.stackexchange.com/questions/329790/how-to-merge-multiple-hard-drives (https://unix.stackexchange.com/questions/329790/how-to-merge-multiple-hard-drives)

Additional info click through on the above reply - https://www.howtoforge.com/linux_lvm (https://www.howtoforge.com/linux_lvm)


Many thanks btsnpcs - very useful information.
The only reservation I have now, after reading the links you gave, is that if one of the drives (within the larger spanned drive) fails, then you risk losing all the data - a significant consideration !
Therefore, bearing this in mind,  the spanned drive would need to be backed-up frequently to mitigate against such data loss occurring ...

Many thanks again - much appreciated.
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: justme2 on January 13, 2018, 04:38:11 PM
Many thanks btsnpcs - very useful information.
The only reservation I have now, after reading the links you gave, is that if one of the drives (within the larger spanned drive) fails, then you risk losing all the data - a significant consideration !
Therefore, bearing this in mind,  the spanned drive would need to be backed-up frequently to mitigate against such data loss occurring ...

Many thanks again - much appreciated.

If a single high capacity HDD fails, the same situation applies - possibility of loss of all data! The question is, is one or more of three spanned drives any more likely to fail than a single drive? Whatever the system, backup is essential but then the backup device may fail. What then? I used CDR/DVR in the past but after storage some became unreadable.

My objection to spanned drives would be the increased noise and power requirements and my strategy would be to convert each drive to USB and use independently.
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: bitsnpcs on January 13, 2018, 07:10:43 PM

Many thanks btsnpcs - very useful information.
The only reservation I have now, after reading the links you gave, is that if one of the drives (within the larger spanned drive) fails, then you risk losing all the data - a significant consideration !
Therefore, bearing this in mind,  the spanned drive would need to be backed-up frequently to mitigate against such data loss occurring ...

Many thanks again - much appreciated.

@m654321 you are right, it is an important consideration when making the decision as to how many to span and how many to use as single backups of specific data as @justme2 writes.
Possibly if there is a drive health tool, like defraggler for Linux, they could be checked, benchmarked and regularly as part of maintenance routine.
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: m654321 on January 14, 2018, 03:46:33 AM
Many thanks btsnpcs - very useful information.
The only reservation I have now, after reading the links you gave, is that if one of the drives (within the larger spanned drive) fails, then you risk losing all the data - a significant consideration !
Therefore, bearing this in mind,  the spanned drive would need to be backed-up frequently to mitigate against such data loss occurring ...

Many thanks again - much appreciated.
If a single high capacity HDD fails, the same situation applies - possibility of loss of all data! The question is, is one or more of three spanned drives any more likely to fail than a single drive?

The answer to that is yes.
The probability or chance of any given single drive failing, no matter how small the risk, is the same, with the caveat of all things being equal: drive type, make etc.
If you have 4 of these drives combined to make 1 larger drive, then the probability of the larger drive failing is increased 4 times, i.e. you sum the probabilities for the risk for each of the component drives.

Any mathematicians out there, please correct me if I'm not making sense on probabilities ... :o

Yes, not very good from an energy consumption point of view either, especially if you have several of these: e.g. around 5 to 10 Amps current drain for 10 drives!

Mike
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: bitsnpcs on January 14, 2018, 08:32:20 AM
I am unsure but I think other factors could be ?

Each file is spread across the 4 hard drives, it is why if one fails the file cannot be fully recovered, does this mean each of the smaller hard drives work less for a file of a given size in comparison to the single large hard drive for a file of the same given size ?

1. on each of the spanned 4 drives -
the arm moves less often and the disk spins less, the clusters used are smaller as the given file is distributed over the 4 in the span (ultimately meaning increased movement, does this yet still = the larger hdd stresses)?

2. Does each individual psu work less whilst spanning and locating due to these factors ?
Is the total of voltage and current drawn for the reduced actions of the four smaller hard drives <,=,> that of the larger single hard drive that works harder for the given large file, in fact will it use the same size psu to begin with, or a scaling be used to calculate this?

3. to determine the factor of heat on the psu whilst working, if it is the same size as the 4 smaller hard drives (or even not) the single large hard drive psu works harder and gets hotter, in theory the larger hard drive psu will fail sooner, due to the effect of heat. If all psu were of the same quality to begin with ?

4. the hard drives are working in a series circuit, like 4 light bulbs in such if one fails they all turn off. If it were possible a radial circuit would be better, then this would not happen as one failing would not have any effect on the others in the circuit.
Radial then it would not be spanned.<< Instead somehow to use switching of multiple hdd that is automated, so the total volume is used/combined, but the individual file is not spread across more than a single hdd, thus if one hdd fails the data on the other 3 is fine, it may also be possible to recover files from the failed hdd in some instances.
That as space is used and filled it begins on the next hdd, if it were intelligent design, it could decide 1 large file would be too much for a remaining capacity of a hdd, and put it on the next hdd, then a later file could fit in the first hdd final capacity so it saves it there, if it is possible ?

5. The power supplies are not in a parallel circuit but their effect on the spanned hard drives are working in the similar manner ?
eg; in that the 4 hard drives overall as a series unit, do receive the same total voltage and could draw the same rate of current (if all psu and hard drives were the same), but not from a single source, as there are 4 psu being used, so the needed parameters are spread across 4 psu each having less load on them and less heat produced.
That is each hard drive has the smallest and largest possibility of the psu available to it.
v/s
The single large hdd psu will work harder and get hotter, and will fail sooner than any of the 4 psu of the spanned hdd ?

6. If 1 thru 5 are measured, calculated, then the probabilities of 4 hard drives v/s 1 can be used like so -
there will be a "sweet number", that is a number of spanned hard drives that prove to be more reliable than a single hard drive of a given total volume, where all numbers of spanned hard drives below that have lower success rates compared to a single large hard drive, and a range above the sweet number it has no improved results over a single hard drive, then upwards from that reaching a final number where the spanned hard drives become more and more unreliable per hard drive added when compared to the single large volume, or maybe not ? ;)
That result would give the best total hard drive size spanned v/s it's equivalent single hard drive.

Update -

I just read this link "Option Three - RAID Array" - https://lifehacker.com/5986883/how-to-combine-multiple-hard-drives-into-one-volume-for-cheap-high-capacity-storage

RAID 10 or RAID 5 looks to solve the issue of hdd failure, I'm not sure how the system is built, I'll look later for some info on it.
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: justme2 on January 14, 2018, 11:13:12 AM
This statement appears towards the end of the document  "Option Three - RAID Array" which you quoted.

"Cons: Unfortunately, RAID has a few downsides too. To start, higher levels of RAID can be expensive, since you need quite a few drives to pull it off. They'd also have to be the same size and speed, or you'd have to sacrifice some of their size and speed. That is, in a RAID array, you're stuck using the smallest disk's capacity for each drive, and the slowest disk's speed for each drive. As such, it isn't ideal if you have a bunch of disks lying around and you're trying to save some money. But, if your data is important, it can help keep that data safe, too. RAID 0's biggest downside is not only the lack of redundancy, but the fact that if you lose one drive, you lose all your data—with little to no hope of recovering it."

Not worth considering IMHO - just convert the drives to USB and connect as required. I'm sure you don't need
multi terrabytes of data all at once. Easier to backup too.  ;)
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: m654321 on January 14, 2018, 01:08:36 PM
This statement appears towards the end of the document  "Option Three - RAID Array" which you quoted.

"Cons: Unfortunately, RAID has a few downsides too. To start, higher levels of RAID can be expensive, since you need quite a few drives to pull it off. They'd also have to be the same size and speed, or you'd have to sacrifice some of their size and speed. That is, in a RAID array, you're stuck using the smallest disk's capacity for each drive, and the slowest disk's speed for each drive. As such, it isn't ideal if you have a bunch of disks lying around and you're trying to save some money. But, if your data is important, it can help keep that data safe, too. RAID 0's biggest downside is not only the lack of redundancy, but the fact that if you lose one drive, you lose all your data—with little to no hope of recovering it."

Not worth considering IMHO - just convert the drives to USB and connect as required. I'm sure you don't need
multi terrabytes of data all at once. Easier to backup too.  ;)

Many thanks for your common sense overview.  I think with all that has been said here on this thread, I'm fast coming to the conclusion it might be simpler, easier & more secure just to buy & use a single larger USB drive for all my video data which is now around 3TB. I have a 4TB USB Expansion Portable Drive (3.6TB formatted drivespace), which I have been very pleased with. A 5TB one has come out, but I might sit tight until an 8TB one comes out, though I think I might be waiting for a while ...

Thanks everybody for your interest in this thread and for the interesting & useful info & links supplied.

Mike 
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: m654321 on January 14, 2018, 01:09:54 PM
I am unsure but I think other factors could be ?

Each file is spread across the 4 hard drives, it is why if one fails the file cannot be fully recovered, does this mean each of the smaller hard drives work less for a file of a given size in comparison to the single large hard drive for a file of the same given size ?

1. on each of the spanned 4 drives -
the arm moves less often and the disk spins less, the clusters used are smaller as the given file is distributed over the 4 in the span (ultimately meaning increased movement, does this yet still = the larger hdd stresses)?

2. Does each individual psu work less whilst spanning and locating due to these factors ?
Is the total of voltage and current drawn for the reduced actions of the four smaller hard drives <,=,> that of the larger single hard drive that works harder for the given large file, in fact will it use the same size psu to begin with, or a scaling be used to calculate this?

3. to determine the factor of heat on the psu whilst working, if it is the same size as the 4 smaller hard drives (or even not) the single large hard drive psu works harder and gets hotter, in theory the larger hard drive psu will fail sooner, due to the effect of heat. If all psu were of the same quality to begin with ?

4. the hard drives are working in a series circuit, like 4 light bulbs in such if one fails they all turn off. If it were possible a radial circuit would be better, then this would not happen as one failing would not have any effect on the others in the circuit.
Radial then it would not be spanned.<< Instead somehow to use switching of multiple hdd that is automated, so the total volume is used/combined, but the individual file is not spread across more than a single hdd, thus if one hdd fails the data on the other 3 is fine, it may also be possible to recover files from the failed hdd in some instances.
That as space is used and filled it begins on the next hdd, if it were intelligent design, it could decide 1 large file would be too much for a remaining capacity of a hdd, and put it on the next hdd, then a later file could fit in the first hdd final capacity so it saves it there, if it is possible ?

5. The power supplies are not in a parallel circuit but their effect on the spanned hard drives are working in the similar manner ?
eg; in that the 4 hard drives overall as a series unit, do receive the same total voltage and could draw the same rate of current (if all psu and hard drives were the same), but not from a single source, as there are 4 psu being used, so the needed parameters are spread across 4 psu each having less load on them and less heat produced.
That is each hard drive has the smallest and largest possibility of the psu available to it.
v/s
The single large hdd psu will work harder and get hotter, and will fail sooner than any of the 4 psu of the spanned hdd ?

6. If 1 thru 5 are measured, calculated, then the probabilities of 4 hard drives v/s 1 can be used like so -
there will be a "sweet number", that is a number of spanned hard drives that prove to be more reliable than a single hard drive of a given total volume, where all numbers of spanned hard drives below that have lower success rates compared to a single large hard drive, and a range above the sweet number it has no improved results over a single hard drive, then upwards from that reaching a final number where the spanned hard drives become more and more unreliable per hard drive added when compared to the single large volume, or maybe not ? ;)
That result would give the best total hard drive size spanned v/s it's equivalent single hard drive.

Update -

I just read this link "Option Three - RAID Array" - https://lifehacker.com/5986883/how-to-combine-multiple-hard-drives-into-one-volume-for-cheap-high-capacity-storage

RAID 10 or RAID 5 looks to solve the issue of hdd failure, I'm not sure how the system is built, I'll look later for some info on it.

Wow, some great work here bitsnpcs - thankyou so much for your thoughts and effort you put into your reply. Really appreciated

Mike
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: bitsnpcs on January 14, 2018, 05:40:52 PM
This statement appears towards the end of the document  "Option Three - RAID Array" which you quoted.

"Cons: Unfortunately, RAID has a few downsides too. To start, higher levels of RAID can be expensive, since you need quite a few drives to pull it off. They'd also have to be the same size and speed, or you'd have to sacrifice some of their size and speed. That is, in a RAID array, you're stuck using the smallest disk's capacity for each drive, and the slowest disk's speed for each drive. As such, it isn't ideal if you have a bunch of disks lying around and you're trying to save some money. But, if your data is important, it can help keep that data safe, too. RAID 0's biggest downside is not only the lack of redundancy, but the fact that if you lose one drive, you lose all your data—with little to no hope of recovering it."

Not worth considering IMHO - just convert the drives to USB and connect as required. I'm sure you don't need
multi terrabytes of data all at once. Easier to backup too.  ;)

No I don't need much space at all.
Title: Re: How do I combine several small HDDs to make one large HDD?
Post by: bitsnpcs on January 14, 2018, 05:53:04 PM
Wow, some great work here bitsnpcs - thankyou so much for your thoughts and effort you put into your reply. Really appreciated

Mike

I enjoyed your thread it was interesting for me seeing/reading lots of new things/possibilities whilst looking around about things.