You are Here:
Linux Lite 6.6 FINAL Released - Support for 22 Languages Added - See Release Announcement Section



How do I combine several small HDDs to make one large HDD?

Author (Read 17232 times)

0 Members and 1 Guest are viewing this topic.

Re: How do I combine several small HDDs to make one large HDD?
« Reply #16 on: January 14, 2018, 05:53:04 PM »
 

bitsnpcs

  • Platinum Level Poster
  • **********
  • 3237
    Posts
  • Reputation: 305
    • View Profile
    • Try to Grow

  • Kernel: 4.x
Wow, some great work here bitsnpcs - thankyou so much for your thoughts and effort you put into your reply. Really appreciated

Mike

I enjoyed your thread it was interesting for me seeing/reading lots of new things/possibilities whilst looking around about things.
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #15 on: January 14, 2018, 05:40:52 PM »
 

bitsnpcs

  • Platinum Level Poster
  • **********
  • 3237
    Posts
  • Reputation: 305
    • View Profile
    • Try to Grow

  • Kernel: 4.x
This statement appears towards the end of the document  "Option Three - RAID Array" which you quoted.

"Cons: Unfortunately, RAID has a few downsides too. To start, higher levels of RAID can be expensive, since you need quite a few drives to pull it off. They'd also have to be the same size and speed, or you'd have to sacrifice some of their size and speed. That is, in a RAID array, you're stuck using the smallest disk's capacity for each drive, and the slowest disk's speed for each drive. As such, it isn't ideal if you have a bunch of disks lying around and you're trying to save some money. But, if your data is important, it can help keep that data safe, too. RAID 0's biggest downside is not only the lack of redundancy, but the fact that if you lose one drive, you lose all your data—with little to no hope of recovering it."

Not worth considering IMHO - just convert the drives to USB and connect as required. I'm sure you don't need
multi terrabytes of data all at once. Easier to backup too.  ;)

No I don't need much space at all.
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #14 on: January 14, 2018, 01:09:54 PM »
 

m654321

  • Gold Level Poster
  • *******
  • 893
    Posts
  • Reputation: 86
  • Linux Lite Member, 'Advocate' & Donator
    • View Profile

  • CPU: Intel Pentium [email protected] (2cores) on an Asus X71Q

  • MEMORY: 4Gb

  • VIDEO CARD: Intel GM45 Express Chipset

  • Kernel: 4.x
I am unsure but I think other factors could be ?

Each file is spread across the 4 hard drives, it is why if one fails the file cannot be fully recovered, does this mean each of the smaller hard drives work less for a file of a given size in comparison to the single large hard drive for a file of the same given size ?

1. on each of the spanned 4 drives -
the arm moves less often and the disk spins less, the clusters used are smaller as the given file is distributed over the 4 in the span (ultimately meaning increased movement, does this yet still = the larger hdd stresses)?

2. Does each individual psu work less whilst spanning and locating due to these factors ?
Is the total of voltage and current drawn for the reduced actions of the four smaller hard drives <,=,> that of the larger single hard drive that works harder for the given large file, in fact will it use the same size psu to begin with, or a scaling be used to calculate this?

3. to determine the factor of heat on the psu whilst working, if it is the same size as the 4 smaller hard drives (or even not) the single large hard drive psu works harder and gets hotter, in theory the larger hard drive psu will fail sooner, due to the effect of heat. If all psu were of the same quality to begin with ?

4. the hard drives are working in a series circuit, like 4 light bulbs in such if one fails they all turn off. If it were possible a radial circuit would be better, then this would not happen as one failing would not have any effect on the others in the circuit.
Radial then it would not be spanned.<< Instead somehow to use switching of multiple hdd that is automated, so the total volume is used/combined, but the individual file is not spread across more than a single hdd, thus if one hdd fails the data on the other 3 is fine, it may also be possible to recover files from the failed hdd in some instances.
That as space is used and filled it begins on the next hdd, if it were intelligent design, it could decide 1 large file would be too much for a remaining capacity of a hdd, and put it on the next hdd, then a later file could fit in the first hdd final capacity so it saves it there, if it is possible ?

5. The power supplies are not in a parallel circuit but their effect on the spanned hard drives are working in the similar manner ?
eg; in that the 4 hard drives overall as a series unit, do receive the same total voltage and could draw the same rate of current (if all psu and hard drives were the same), but not from a single source, as there are 4 psu being used, so the needed parameters are spread across 4 psu each having less load on them and less heat produced.
That is each hard drive has the smallest and largest possibility of the psu available to it.
v/s
The single large hdd psu will work harder and get hotter, and will fail sooner than any of the 4 psu of the spanned hdd ?

6. If 1 thru 5 are measured, calculated, then the probabilities of 4 hard drives v/s 1 can be used like so -
there will be a "sweet number", that is a number of spanned hard drives that prove to be more reliable than a single hard drive of a given total volume, where all numbers of spanned hard drives below that have lower success rates compared to a single large hard drive, and a range above the sweet number it has no improved results over a single hard drive, then upwards from that reaching a final number where the spanned hard drives become more and more unreliable per hard drive added when compared to the single large volume, or maybe not ? ;)
That result would give the best total hard drive size spanned v/s it's equivalent single hard drive.

Update -

I just read this link "Option Three - RAID Array" - https://lifehacker.com/5986883/how-to-combine-multiple-hard-drives-into-one-volume-for-cheap-high-capacity-storage

RAID 10 or RAID 5 looks to solve the issue of hdd failure, I'm not sure how the system is built, I'll look later for some info on it.

Wow, some great work here bitsnpcs - thankyou so much for your thoughts and effort you put into your reply. Really appreciated

Mike
64bit OS (32-bit on Samsung netbook) installed in Legacy mode on MBR-formatted SSDs (except pi which uses a micro SDHC card):
2017 - Raspberry pi 3B (4cores) ~ [email protected] - LibreElec, used for upgrading our Samsung TV (excellent for the task)  
2012 - Lenovo G580 2689 (2cores; 4threads] ~ [email protected] - LL3.8/Win8.1 dual-boot (LL working smoothly)
2011 - Samsung NP-N145 Plus (1core; 2threads) ~ Intel Atom [email protected] - LL 3.8 32-bit (64-bit too 'laggy')
2008 - Asus X71Q (2cores) ~ Intel [email protected] - LL4.6/Win8.1 dual-boot, LL works fine with kernel 4.15
2007 - Dell Latitude D630 (2cores) ~ Intel [email protected] - LL4.6, works well with kernel 4.4; 4.15 doesn't work
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #13 on: January 14, 2018, 01:08:36 PM »
 

m654321

  • Gold Level Poster
  • *******
  • 893
    Posts
  • Reputation: 86
  • Linux Lite Member, 'Advocate' & Donator
    • View Profile

  • CPU: Intel Pentium [email protected] (2cores) on an Asus X71Q

  • MEMORY: 4Gb

  • VIDEO CARD: Intel GM45 Express Chipset

  • Kernel: 4.x
This statement appears towards the end of the document  "Option Three - RAID Array" which you quoted.

"Cons: Unfortunately, RAID has a few downsides too. To start, higher levels of RAID can be expensive, since you need quite a few drives to pull it off. They'd also have to be the same size and speed, or you'd have to sacrifice some of their size and speed. That is, in a RAID array, you're stuck using the smallest disk's capacity for each drive, and the slowest disk's speed for each drive. As such, it isn't ideal if you have a bunch of disks lying around and you're trying to save some money. But, if your data is important, it can help keep that data safe, too. RAID 0's biggest downside is not only the lack of redundancy, but the fact that if you lose one drive, you lose all your data—with little to no hope of recovering it."

Not worth considering IMHO - just convert the drives to USB and connect as required. I'm sure you don't need
multi terrabytes of data all at once. Easier to backup too.  ;)

Many thanks for your common sense overview.  I think with all that has been said here on this thread, I'm fast coming to the conclusion it might be simpler, easier & more secure just to buy & use a single larger USB drive for all my video data which is now around 3TB. I have a 4TB USB Expansion Portable Drive (3.6TB formatted drivespace), which I have been very pleased with. A 5TB one has come out, but I might sit tight until an 8TB one comes out, though I think I might be waiting for a while ...

Thanks everybody for your interest in this thread and for the interesting & useful info & links supplied.

Mike 
64bit OS (32-bit on Samsung netbook) installed in Legacy mode on MBR-formatted SSDs (except pi which uses a micro SDHC card):
2017 - Raspberry pi 3B (4cores) ~ [email protected] - LibreElec, used for upgrading our Samsung TV (excellent for the task)  
2012 - Lenovo G580 2689 (2cores; 4threads] ~ [email protected] - LL3.8/Win8.1 dual-boot (LL working smoothly)
2011 - Samsung NP-N145 Plus (1core; 2threads) ~ Intel Atom [email protected] - LL 3.8 32-bit (64-bit too 'laggy')
2008 - Asus X71Q (2cores) ~ Intel [email protected] - LL4.6/Win8.1 dual-boot, LL works fine with kernel 4.15
2007 - Dell Latitude D630 (2cores) ~ Intel [email protected] - LL4.6, works well with kernel 4.4; 4.15 doesn't work
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #12 on: January 14, 2018, 11:13:12 AM »
 

justme2

  • PayPal Supporter
  • Forum Regular
  • *****
  • 201
    Posts
  • Reputation: 25
  • Linux Lite Member
    • View Profile

  • CPU: Intel i5 vPro

  • MEMORY: 8Gb

  • VIDEO CARD: Intel
This statement appears towards the end of the document  "Option Three - RAID Array" which you quoted.

"Cons: Unfortunately, RAID has a few downsides too. To start, higher levels of RAID can be expensive, since you need quite a few drives to pull it off. They'd also have to be the same size and speed, or you'd have to sacrifice some of their size and speed. That is, in a RAID array, you're stuck using the smallest disk's capacity for each drive, and the slowest disk's speed for each drive. As such, it isn't ideal if you have a bunch of disks lying around and you're trying to save some money. But, if your data is important, it can help keep that data safe, too. RAID 0's biggest downside is not only the lack of redundancy, but the fact that if you lose one drive, you lose all your data—with little to no hope of recovering it."

Not worth considering IMHO - just convert the drives to USB and connect as required. I'm sure you don't need
multi terrabytes of data all at once. Easier to backup too.  ;)
1) Lenovo T520 i5 LL3.8 8GB ram, fast & stable
2) Medion P4 32bit LL3.8 1GB ram, quite fast & stable
3) eeePC 901 32bit LL3.8 1GB ram, fast & stable
4) eeePC 701 32bit LL3.8 1GB ram, slower & stable but small and light enough to travel with me to New Zealand when visiting family in Blenheim.
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #11 on: January 14, 2018, 08:32:20 AM »
 

bitsnpcs

  • Platinum Level Poster
  • **********
  • 3237
    Posts
  • Reputation: 305
    • View Profile
    • Try to Grow

  • Kernel: 4.x
I am unsure but I think other factors could be ?

Each file is spread across the 4 hard drives, it is why if one fails the file cannot be fully recovered, does this mean each of the smaller hard drives work less for a file of a given size in comparison to the single large hard drive for a file of the same given size ?

1. on each of the spanned 4 drives -
the arm moves less often and the disk spins less, the clusters used are smaller as the given file is distributed over the 4 in the span (ultimately meaning increased movement, does this yet still = the larger hdd stresses)?

2. Does each individual psu work less whilst spanning and locating due to these factors ?
Is the total of voltage and current drawn for the reduced actions of the four smaller hard drives <,=,> that of the larger single hard drive that works harder for the given large file, in fact will it use the same size psu to begin with, or a scaling be used to calculate this?

3. to determine the factor of heat on the psu whilst working, if it is the same size as the 4 smaller hard drives (or even not) the single large hard drive psu works harder and gets hotter, in theory the larger hard drive psu will fail sooner, due to the effect of heat. If all psu were of the same quality to begin with ?

4. the hard drives are working in a series circuit, like 4 light bulbs in such if one fails they all turn off. If it were possible a radial circuit would be better, then this would not happen as one failing would not have any effect on the others in the circuit.
Radial then it would not be spanned.<< Instead somehow to use switching of multiple hdd that is automated, so the total volume is used/combined, but the individual file is not spread across more than a single hdd, thus if one hdd fails the data on the other 3 is fine, it may also be possible to recover files from the failed hdd in some instances.
That as space is used and filled it begins on the next hdd, if it were intelligent design, it could decide 1 large file would be too much for a remaining capacity of a hdd, and put it on the next hdd, then a later file could fit in the first hdd final capacity so it saves it there, if it is possible ?

5. The power supplies are not in a parallel circuit but their effect on the spanned hard drives are working in the similar manner ?
eg; in that the 4 hard drives overall as a series unit, do receive the same total voltage and could draw the same rate of current (if all psu and hard drives were the same), but not from a single source, as there are 4 psu being used, so the needed parameters are spread across 4 psu each having less load on them and less heat produced.
That is each hard drive has the smallest and largest possibility of the psu available to it.
v/s
The single large hdd psu will work harder and get hotter, and will fail sooner than any of the 4 psu of the spanned hdd ?

6. If 1 thru 5 are measured, calculated, then the probabilities of 4 hard drives v/s 1 can be used like so -
there will be a "sweet number", that is a number of spanned hard drives that prove to be more reliable than a single hard drive of a given total volume, where all numbers of spanned hard drives below that have lower success rates compared to a single large hard drive, and a range above the sweet number it has no improved results over a single hard drive, then upwards from that reaching a final number where the spanned hard drives become more and more unreliable per hard drive added when compared to the single large volume, or maybe not ? ;)
That result would give the best total hard drive size spanned v/s it's equivalent single hard drive.

Update -

I just read this link "Option Three - RAID Array" - https://lifehacker.com/5986883/how-to-combine-multiple-hard-drives-into-one-volume-for-cheap-high-capacity-storage

RAID 10 or RAID 5 looks to solve the issue of hdd failure, I'm not sure how the system is built, I'll look later for some info on it.
« Last Edit: January 14, 2018, 08:59:23 AM by bitsnpcs »
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #10 on: January 14, 2018, 03:46:33 AM »
 

m654321

  • Gold Level Poster
  • *******
  • 893
    Posts
  • Reputation: 86
  • Linux Lite Member, 'Advocate' & Donator
    • View Profile

  • CPU: Intel Pentium [email protected] (2cores) on an Asus X71Q

  • MEMORY: 4Gb

  • VIDEO CARD: Intel GM45 Express Chipset

  • Kernel: 4.x
Many thanks btsnpcs - very useful information.
The only reservation I have now, after reading the links you gave, is that if one of the drives (within the larger spanned drive) fails, then you risk losing all the data - a significant consideration !
Therefore, bearing this in mind,  the spanned drive would need to be backed-up frequently to mitigate against such data loss occurring ...

Many thanks again - much appreciated.
If a single high capacity HDD fails, the same situation applies - possibility of loss of all data! The question is, is one or more of three spanned drives any more likely to fail than a single drive?

The answer to that is yes.
The probability or chance of any given single drive failing, no matter how small the risk, is the same, with the caveat of all things being equal: drive type, make etc.
If you have 4 of these drives combined to make 1 larger drive, then the probability of the larger drive failing is increased 4 times, i.e. you sum the probabilities for the risk for each of the component drives.

Any mathematicians out there, please correct me if I'm not making sense on probabilities ... :o

Yes, not very good from an energy consumption point of view either, especially if you have several of these: e.g. around 5 to 10 Amps current drain for 10 drives!

Mike
« Last Edit: January 14, 2018, 03:53:23 AM by m654321 »
64bit OS (32-bit on Samsung netbook) installed in Legacy mode on MBR-formatted SSDs (except pi which uses a micro SDHC card):
2017 - Raspberry pi 3B (4cores) ~ [email protected] - LibreElec, used for upgrading our Samsung TV (excellent for the task)  
2012 - Lenovo G580 2689 (2cores; 4threads] ~ [email protected] - LL3.8/Win8.1 dual-boot (LL working smoothly)
2011 - Samsung NP-N145 Plus (1core; 2threads) ~ Intel Atom [email protected] - LL 3.8 32-bit (64-bit too 'laggy')
2008 - Asus X71Q (2cores) ~ Intel [email protected] - LL4.6/Win8.1 dual-boot, LL works fine with kernel 4.15
2007 - Dell Latitude D630 (2cores) ~ Intel [email protected] - LL4.6, works well with kernel 4.4; 4.15 doesn't work
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #9 on: January 13, 2018, 07:10:43 PM »
 

bitsnpcs

  • Platinum Level Poster
  • **********
  • 3237
    Posts
  • Reputation: 305
    • View Profile
    • Try to Grow

  • Kernel: 4.x

Many thanks btsnpcs - very useful information.
The only reservation I have now, after reading the links you gave, is that if one of the drives (within the larger spanned drive) fails, then you risk losing all the data - a significant consideration !
Therefore, bearing this in mind,  the spanned drive would need to be backed-up frequently to mitigate against such data loss occurring ...

Many thanks again - much appreciated.

@m654321 you are right, it is an important consideration when making the decision as to how many to span and how many to use as single backups of specific data as @justme2 writes.
Possibly if there is a drive health tool, like defraggler for Linux, they could be checked, benchmarked and regularly as part of maintenance routine.
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #8 on: January 13, 2018, 04:38:11 PM »
 

justme2

  • PayPal Supporter
  • Forum Regular
  • *****
  • 201
    Posts
  • Reputation: 25
  • Linux Lite Member
    • View Profile

  • CPU: Intel i5 vPro

  • MEMORY: 8Gb

  • VIDEO CARD: Intel
Many thanks btsnpcs - very useful information.
The only reservation I have now, after reading the links you gave, is that if one of the drives (within the larger spanned drive) fails, then you risk losing all the data - a significant consideration !
Therefore, bearing this in mind,  the spanned drive would need to be backed-up frequently to mitigate against such data loss occurring ...

Many thanks again - much appreciated.

If a single high capacity HDD fails, the same situation applies - possibility of loss of all data! The question is, is one or more of three spanned drives any more likely to fail than a single drive? Whatever the system, backup is essential but then the backup device may fail. What then? I used CDR/DVR in the past but after storage some became unreadable.

My objection to spanned drives would be the increased noise and power requirements and my strategy would be to convert each drive to USB and use independently.
1) Lenovo T520 i5 LL3.8 8GB ram, fast & stable
2) Medion P4 32bit LL3.8 1GB ram, quite fast & stable
3) eeePC 901 32bit LL3.8 1GB ram, fast & stable
4) eeePC 701 32bit LL3.8 1GB ram, slower & stable but small and light enough to travel with me to New Zealand when visiting family in Blenheim.
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #7 on: January 13, 2018, 02:59:46 PM »
 

m654321

  • Gold Level Poster
  • *******
  • 893
    Posts
  • Reputation: 86
  • Linux Lite Member, 'Advocate' & Donator
    • View Profile

  • CPU: Intel Pentium [email protected] (2cores) on an Asus X71Q

  • MEMORY: 4Gb

  • VIDEO CARD: Intel GM45 Express Chipset

  • Kernel: 4.x
Hello Mike,

for Linux - https://unix.stackexchange.com/questions/329790/how-to-merge-multiple-hard-drives

Additional info click through on the above reply - https://www.howtoforge.com/linux_lvm


Many thanks btsnpcs - very useful information.
The only reservation I have now, after reading the links you gave, is that if one of the drives (within the larger spanned drive) fails, then you risk losing all the data - a significant consideration !
Therefore, bearing this in mind,  the spanned drive would need to be backed-up frequently to mitigate against such data loss occurring ...

Many thanks again - much appreciated.
« Last Edit: January 13, 2018, 03:05:31 PM by m654321 »
64bit OS (32-bit on Samsung netbook) installed in Legacy mode on MBR-formatted SSDs (except pi which uses a micro SDHC card):
2017 - Raspberry pi 3B (4cores) ~ [email protected] - LibreElec, used for upgrading our Samsung TV (excellent for the task)  
2012 - Lenovo G580 2689 (2cores; 4threads] ~ [email protected] - LL3.8/Win8.1 dual-boot (LL working smoothly)
2011 - Samsung NP-N145 Plus (1core; 2threads) ~ Intel Atom [email protected] - LL 3.8 32-bit (64-bit too 'laggy')
2008 - Asus X71Q (2cores) ~ Intel [email protected] - LL4.6/Win8.1 dual-boot, LL works fine with kernel 4.15
2007 - Dell Latitude D630 (2cores) ~ Intel [email protected] - LL4.6, works well with kernel 4.4; 4.15 doesn't work
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #6 on: January 12, 2018, 09:05:12 PM »
 

bitsnpcs

  • Platinum Level Poster
  • **********
  • 3237
    Posts
  • Reputation: 305
    • View Profile
    • Try to Grow

  • Kernel: 4.x
@Moltke if many spare hdd combined -






It seems there can be made "pools", a combined group effort, it could be done by a community, as a source of funding for a project/s.
« Last Edit: January 12, 2018, 09:09:20 PM by bitsnpcs »
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #5 on: January 12, 2018, 07:09:46 PM »
 

Moltke

  • Platinum Level Poster
  • **********
  • 1134
    Posts
  • Reputation: 126
  • Linux Lite Member
    • View Profile

  • CPU: amd athlon 64 x2

  • MEMORY: 4Gb

  • VIDEO CARD: amd radeon hd 6750

  • Kernel: 5.x
@bitsnpcs  Such a nice informaton! I didn't even ever think this was possible. Bookmarked!
Without each others help there ain't no hope for us :)
Need a translation service? https://www.deepl.com/es/translator
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #4 on: January 12, 2018, 11:51:29 AM »
 

bitsnpcs

  • Platinum Level Poster
  • **********
  • 3237
    Posts
  • Reputation: 305
    • View Profile
    • Try to Grow

  • Kernel: 4.x
@Scott I'll try it too when I replace the 32 bit desktops this year.



 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #3 on: January 12, 2018, 10:22:18 AM »
 

Scott

  • Global Moderator
  • Gold Level Poster
  • *****
  • 857
    Posts
  • Reputation: 186
  • Linux Lite Member
    • View Profile

  • CPU: Dual core Intel Core i3 M 330

  • MEMORY: 6Gb

  • VIDEO CARD: Intel Integrated Graphics
@bitsnpcs - Very good information, thank you.
 

Re: How do I combine several small HDDs to make one large HDD?
« Reply #2 on: January 12, 2018, 09:21:39 AM »
 

bitsnpcs

  • Platinum Level Poster
  • **********
  • 3237
    Posts
  • Reputation: 305
    • View Profile
    • Try to Grow

  • Kernel: 4.x
Hello Mike,

for Linux - https://unix.stackexchange.com/questions/329790/how-to-merge-multiple-hard-drives

Additional info click through on the above reply - https://www.howtoforge.com/linux_lvm


If you also wish to use it on windows machines


« Last Edit: January 12, 2018, 09:24:12 AM by bitsnpcs »
 

 

-->
X Close Ad

Linux Lite 6.6 FINAL Released - Support for 22 Languages Added - See Release Announcement Section