Amazon AWS Ephemeral disks and RAID1











up vote
3
down vote

favorite












Some AWS instances have "ephemeral disks" attached, which are much faster than EBS. But ephemeral disks will be blank and uninitialised when your instance is stopped and started. The data on disk generally survives an instance reboot though.



Question: Should I use a software RAID1 on my AWS instance, built over an ephemeral disk and an EBS volume?



My thinking is that the raid1 will come up in degraded mode with the EBS volume only, and then we can use mdadm commands to add the blank ephemeral disk back into the raid. This will allow the app to start up 5-10 minutes sooner, at the cost of worse performance while the raid1 synchronises.



Background: I have an app that uses ~40 GB of data files. Access times directly corellate with performance, so faster the disk the faster the app works.



Historically we've run something from rc.local to rsync data from an EBS disk to the ephemeral disk, and then started the software. The synch takes 5-10 minutes, better than the 5-20 minutes it took to synch from another instance. In the past we've even used the data files from a ramdisk, which was not as fast as the ephemeral disks.





More info - this is an i3.4xlarge so it has 2x NVME ephemeral drives.



# hdparm -t /dev/md? /dev/nvme?n1 /dev/xvd?
/dev/md0: 9510 MB in 3.00 seconds = 3169.78 MB/sec RAID0 of two eph drives
/dev/nvme0n1: 4008 MB in 3.00 seconds = 1335.74 MB/sec Eph drive
/dev/nvme1n1: 4014 MB in 3.00 seconds = 1337.48 MB/sec Eph drive
/dev/xvda: 524 MB in 3.01 seconds = 174.17 MB/sec gp2 16GB, 100 IOPs root
/dev/xvdf: 524 MB in 3.01 seconds = 174.23 MB/sec gp2 120GB, 300 IOPs data
/dev/xvdz: 874 MB in 3.01 seconds = 290.68 MB/sec gp2 500GB, 1500 IOPs raid-seed disk


I have created a raid1 with



mdadm  --create /dev/md1 --raid-devices=3 --verbose --level=1 /dev/nvme?n1 /dev/xvdz


which returns:



$ cat /proc/mdstat
Personalities : [raid0] [raid1]
md1 : active raid1 nvme1n1[4] nvme0n1[3] xvdz[2]
524155904 blocks super 1.2 [3/3] [UUU]
bitmap: 0/4 pages [0KB], 65536KB chunk


Curiously, the raid reads about as fast as the faster drives, and is not limited to the speed of the slowest disk.



/dev/md1:     4042 MB in  3.00 seconds = 1346.67 MB/sec
/dev/nvme0n1: 4104 MB in 3.00 seconds = 1367.62 MB/sec
/dev/nvme1n1: 4030 MB in 3.00 seconds = 1342.93 MB/sec
/dev/xvdz: 668 MB in 3.01 seconds = 222.26 MB/sec


A power-off/on returns a degraded raidset, but the app can still run albeit slower. The cost of waiting 5-10 minutes is avoided, and I can re-add the ephemeral disks to the raid on the fly without an app restart.



So while it seems to work perfectly, is there anything I've missed or not considered?










share|improve this question




















  • 1




    How often does the data on your disk change? What's the RTO and RPO? Interesting idea to span RAID across them, but it seems a bit "hacky" and I wonder if there's a better solution. EFS with some kind of disk cache perhaps, a script to populate the ephemeral disk from EBS, that kind of thing
    – Tim
    4 hours ago










  • @tim data files are updated once every 3 months, and is read-only by the application. Recovery time isn't particularly important, the host is redundant, however the app just runs slower as disks get slower, which is why fastest-disk-possible is important. We already have a hacky script to populate from EBS. I'm spinning this up at the moment and will provide some timing tests soon.
    – Criggie
    3 hours ago










  • Further, with a raid0 of the two eph disks, the application's performance metric is ~1345 ms. With a raid1 of two eph disks plus an EBS disk, it gets 914 ~ms So its running better than before.
    – Criggie
    1 hour ago










  • In a Raid1 it might speed up your read load, but nut your write load. An alternative would be a Read Cache (blockcaache/lvmcache). That’s probably better to control policy and has no rebuild penalty.
    – eckes
    35 mins ago















up vote
3
down vote

favorite












Some AWS instances have "ephemeral disks" attached, which are much faster than EBS. But ephemeral disks will be blank and uninitialised when your instance is stopped and started. The data on disk generally survives an instance reboot though.



Question: Should I use a software RAID1 on my AWS instance, built over an ephemeral disk and an EBS volume?



My thinking is that the raid1 will come up in degraded mode with the EBS volume only, and then we can use mdadm commands to add the blank ephemeral disk back into the raid. This will allow the app to start up 5-10 minutes sooner, at the cost of worse performance while the raid1 synchronises.



Background: I have an app that uses ~40 GB of data files. Access times directly corellate with performance, so faster the disk the faster the app works.



Historically we've run something from rc.local to rsync data from an EBS disk to the ephemeral disk, and then started the software. The synch takes 5-10 minutes, better than the 5-20 minutes it took to synch from another instance. In the past we've even used the data files from a ramdisk, which was not as fast as the ephemeral disks.





More info - this is an i3.4xlarge so it has 2x NVME ephemeral drives.



# hdparm -t /dev/md? /dev/nvme?n1 /dev/xvd?
/dev/md0: 9510 MB in 3.00 seconds = 3169.78 MB/sec RAID0 of two eph drives
/dev/nvme0n1: 4008 MB in 3.00 seconds = 1335.74 MB/sec Eph drive
/dev/nvme1n1: 4014 MB in 3.00 seconds = 1337.48 MB/sec Eph drive
/dev/xvda: 524 MB in 3.01 seconds = 174.17 MB/sec gp2 16GB, 100 IOPs root
/dev/xvdf: 524 MB in 3.01 seconds = 174.23 MB/sec gp2 120GB, 300 IOPs data
/dev/xvdz: 874 MB in 3.01 seconds = 290.68 MB/sec gp2 500GB, 1500 IOPs raid-seed disk


I have created a raid1 with



mdadm  --create /dev/md1 --raid-devices=3 --verbose --level=1 /dev/nvme?n1 /dev/xvdz


which returns:



$ cat /proc/mdstat
Personalities : [raid0] [raid1]
md1 : active raid1 nvme1n1[4] nvme0n1[3] xvdz[2]
524155904 blocks super 1.2 [3/3] [UUU]
bitmap: 0/4 pages [0KB], 65536KB chunk


Curiously, the raid reads about as fast as the faster drives, and is not limited to the speed of the slowest disk.



/dev/md1:     4042 MB in  3.00 seconds = 1346.67 MB/sec
/dev/nvme0n1: 4104 MB in 3.00 seconds = 1367.62 MB/sec
/dev/nvme1n1: 4030 MB in 3.00 seconds = 1342.93 MB/sec
/dev/xvdz: 668 MB in 3.01 seconds = 222.26 MB/sec


A power-off/on returns a degraded raidset, but the app can still run albeit slower. The cost of waiting 5-10 minutes is avoided, and I can re-add the ephemeral disks to the raid on the fly without an app restart.



So while it seems to work perfectly, is there anything I've missed or not considered?










share|improve this question




















  • 1




    How often does the data on your disk change? What's the RTO and RPO? Interesting idea to span RAID across them, but it seems a bit "hacky" and I wonder if there's a better solution. EFS with some kind of disk cache perhaps, a script to populate the ephemeral disk from EBS, that kind of thing
    – Tim
    4 hours ago










  • @tim data files are updated once every 3 months, and is read-only by the application. Recovery time isn't particularly important, the host is redundant, however the app just runs slower as disks get slower, which is why fastest-disk-possible is important. We already have a hacky script to populate from EBS. I'm spinning this up at the moment and will provide some timing tests soon.
    – Criggie
    3 hours ago










  • Further, with a raid0 of the two eph disks, the application's performance metric is ~1345 ms. With a raid1 of two eph disks plus an EBS disk, it gets 914 ~ms So its running better than before.
    – Criggie
    1 hour ago










  • In a Raid1 it might speed up your read load, but nut your write load. An alternative would be a Read Cache (blockcaache/lvmcache). That’s probably better to control policy and has no rebuild penalty.
    – eckes
    35 mins ago













up vote
3
down vote

favorite









up vote
3
down vote

favorite











Some AWS instances have "ephemeral disks" attached, which are much faster than EBS. But ephemeral disks will be blank and uninitialised when your instance is stopped and started. The data on disk generally survives an instance reboot though.



Question: Should I use a software RAID1 on my AWS instance, built over an ephemeral disk and an EBS volume?



My thinking is that the raid1 will come up in degraded mode with the EBS volume only, and then we can use mdadm commands to add the blank ephemeral disk back into the raid. This will allow the app to start up 5-10 minutes sooner, at the cost of worse performance while the raid1 synchronises.



Background: I have an app that uses ~40 GB of data files. Access times directly corellate with performance, so faster the disk the faster the app works.



Historically we've run something from rc.local to rsync data from an EBS disk to the ephemeral disk, and then started the software. The synch takes 5-10 minutes, better than the 5-20 minutes it took to synch from another instance. In the past we've even used the data files from a ramdisk, which was not as fast as the ephemeral disks.





More info - this is an i3.4xlarge so it has 2x NVME ephemeral drives.



# hdparm -t /dev/md? /dev/nvme?n1 /dev/xvd?
/dev/md0: 9510 MB in 3.00 seconds = 3169.78 MB/sec RAID0 of two eph drives
/dev/nvme0n1: 4008 MB in 3.00 seconds = 1335.74 MB/sec Eph drive
/dev/nvme1n1: 4014 MB in 3.00 seconds = 1337.48 MB/sec Eph drive
/dev/xvda: 524 MB in 3.01 seconds = 174.17 MB/sec gp2 16GB, 100 IOPs root
/dev/xvdf: 524 MB in 3.01 seconds = 174.23 MB/sec gp2 120GB, 300 IOPs data
/dev/xvdz: 874 MB in 3.01 seconds = 290.68 MB/sec gp2 500GB, 1500 IOPs raid-seed disk


I have created a raid1 with



mdadm  --create /dev/md1 --raid-devices=3 --verbose --level=1 /dev/nvme?n1 /dev/xvdz


which returns:



$ cat /proc/mdstat
Personalities : [raid0] [raid1]
md1 : active raid1 nvme1n1[4] nvme0n1[3] xvdz[2]
524155904 blocks super 1.2 [3/3] [UUU]
bitmap: 0/4 pages [0KB], 65536KB chunk


Curiously, the raid reads about as fast as the faster drives, and is not limited to the speed of the slowest disk.



/dev/md1:     4042 MB in  3.00 seconds = 1346.67 MB/sec
/dev/nvme0n1: 4104 MB in 3.00 seconds = 1367.62 MB/sec
/dev/nvme1n1: 4030 MB in 3.00 seconds = 1342.93 MB/sec
/dev/xvdz: 668 MB in 3.01 seconds = 222.26 MB/sec


A power-off/on returns a degraded raidset, but the app can still run albeit slower. The cost of waiting 5-10 minutes is avoided, and I can re-add the ephemeral disks to the raid on the fly without an app restart.



So while it seems to work perfectly, is there anything I've missed or not considered?










share|improve this question















Some AWS instances have "ephemeral disks" attached, which are much faster than EBS. But ephemeral disks will be blank and uninitialised when your instance is stopped and started. The data on disk generally survives an instance reboot though.



Question: Should I use a software RAID1 on my AWS instance, built over an ephemeral disk and an EBS volume?



My thinking is that the raid1 will come up in degraded mode with the EBS volume only, and then we can use mdadm commands to add the blank ephemeral disk back into the raid. This will allow the app to start up 5-10 minutes sooner, at the cost of worse performance while the raid1 synchronises.



Background: I have an app that uses ~40 GB of data files. Access times directly corellate with performance, so faster the disk the faster the app works.



Historically we've run something from rc.local to rsync data from an EBS disk to the ephemeral disk, and then started the software. The synch takes 5-10 minutes, better than the 5-20 minutes it took to synch from another instance. In the past we've even used the data files from a ramdisk, which was not as fast as the ephemeral disks.





More info - this is an i3.4xlarge so it has 2x NVME ephemeral drives.



# hdparm -t /dev/md? /dev/nvme?n1 /dev/xvd?
/dev/md0: 9510 MB in 3.00 seconds = 3169.78 MB/sec RAID0 of two eph drives
/dev/nvme0n1: 4008 MB in 3.00 seconds = 1335.74 MB/sec Eph drive
/dev/nvme1n1: 4014 MB in 3.00 seconds = 1337.48 MB/sec Eph drive
/dev/xvda: 524 MB in 3.01 seconds = 174.17 MB/sec gp2 16GB, 100 IOPs root
/dev/xvdf: 524 MB in 3.01 seconds = 174.23 MB/sec gp2 120GB, 300 IOPs data
/dev/xvdz: 874 MB in 3.01 seconds = 290.68 MB/sec gp2 500GB, 1500 IOPs raid-seed disk


I have created a raid1 with



mdadm  --create /dev/md1 --raid-devices=3 --verbose --level=1 /dev/nvme?n1 /dev/xvdz


which returns:



$ cat /proc/mdstat
Personalities : [raid0] [raid1]
md1 : active raid1 nvme1n1[4] nvme0n1[3] xvdz[2]
524155904 blocks super 1.2 [3/3] [UUU]
bitmap: 0/4 pages [0KB], 65536KB chunk


Curiously, the raid reads about as fast as the faster drives, and is not limited to the speed of the slowest disk.



/dev/md1:     4042 MB in  3.00 seconds = 1346.67 MB/sec
/dev/nvme0n1: 4104 MB in 3.00 seconds = 1367.62 MB/sec
/dev/nvme1n1: 4030 MB in 3.00 seconds = 1342.93 MB/sec
/dev/xvdz: 668 MB in 3.01 seconds = 222.26 MB/sec


A power-off/on returns a degraded raidset, but the app can still run albeit slower. The cost of waiting 5-10 minutes is avoided, and I can re-add the ephemeral disks to the raid on the fly without an app restart.



So while it seems to work perfectly, is there anything I've missed or not considered?







amazon-ec2 software-raid amazon-ebs raid1 amazon-ephemeral






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 1 hour ago

























asked 4 hours ago









Criggie

1,056618




1,056618








  • 1




    How often does the data on your disk change? What's the RTO and RPO? Interesting idea to span RAID across them, but it seems a bit "hacky" and I wonder if there's a better solution. EFS with some kind of disk cache perhaps, a script to populate the ephemeral disk from EBS, that kind of thing
    – Tim
    4 hours ago










  • @tim data files are updated once every 3 months, and is read-only by the application. Recovery time isn't particularly important, the host is redundant, however the app just runs slower as disks get slower, which is why fastest-disk-possible is important. We already have a hacky script to populate from EBS. I'm spinning this up at the moment and will provide some timing tests soon.
    – Criggie
    3 hours ago










  • Further, with a raid0 of the two eph disks, the application's performance metric is ~1345 ms. With a raid1 of two eph disks plus an EBS disk, it gets 914 ~ms So its running better than before.
    – Criggie
    1 hour ago










  • In a Raid1 it might speed up your read load, but nut your write load. An alternative would be a Read Cache (blockcaache/lvmcache). That’s probably better to control policy and has no rebuild penalty.
    – eckes
    35 mins ago














  • 1




    How often does the data on your disk change? What's the RTO and RPO? Interesting idea to span RAID across them, but it seems a bit "hacky" and I wonder if there's a better solution. EFS with some kind of disk cache perhaps, a script to populate the ephemeral disk from EBS, that kind of thing
    – Tim
    4 hours ago










  • @tim data files are updated once every 3 months, and is read-only by the application. Recovery time isn't particularly important, the host is redundant, however the app just runs slower as disks get slower, which is why fastest-disk-possible is important. We already have a hacky script to populate from EBS. I'm spinning this up at the moment and will provide some timing tests soon.
    – Criggie
    3 hours ago










  • Further, with a raid0 of the two eph disks, the application's performance metric is ~1345 ms. With a raid1 of two eph disks plus an EBS disk, it gets 914 ~ms So its running better than before.
    – Criggie
    1 hour ago










  • In a Raid1 it might speed up your read load, but nut your write load. An alternative would be a Read Cache (blockcaache/lvmcache). That’s probably better to control policy and has no rebuild penalty.
    – eckes
    35 mins ago








1




1




How often does the data on your disk change? What's the RTO and RPO? Interesting idea to span RAID across them, but it seems a bit "hacky" and I wonder if there's a better solution. EFS with some kind of disk cache perhaps, a script to populate the ephemeral disk from EBS, that kind of thing
– Tim
4 hours ago




How often does the data on your disk change? What's the RTO and RPO? Interesting idea to span RAID across them, but it seems a bit "hacky" and I wonder if there's a better solution. EFS with some kind of disk cache perhaps, a script to populate the ephemeral disk from EBS, that kind of thing
– Tim
4 hours ago












@tim data files are updated once every 3 months, and is read-only by the application. Recovery time isn't particularly important, the host is redundant, however the app just runs slower as disks get slower, which is why fastest-disk-possible is important. We already have a hacky script to populate from EBS. I'm spinning this up at the moment and will provide some timing tests soon.
– Criggie
3 hours ago




@tim data files are updated once every 3 months, and is read-only by the application. Recovery time isn't particularly important, the host is redundant, however the app just runs slower as disks get slower, which is why fastest-disk-possible is important. We already have a hacky script to populate from EBS. I'm spinning this up at the moment and will provide some timing tests soon.
– Criggie
3 hours ago












Further, with a raid0 of the two eph disks, the application's performance metric is ~1345 ms. With a raid1 of two eph disks plus an EBS disk, it gets 914 ~ms So its running better than before.
– Criggie
1 hour ago




Further, with a raid0 of the two eph disks, the application's performance metric is ~1345 ms. With a raid1 of two eph disks plus an EBS disk, it gets 914 ~ms So its running better than before.
– Criggie
1 hour ago












In a Raid1 it might speed up your read load, but nut your write load. An alternative would be a Read Cache (blockcaache/lvmcache). That’s probably better to control policy and has no rebuild penalty.
– eckes
35 mins ago




In a Raid1 it might speed up your read load, but nut your write load. An alternative would be a Read Cache (blockcaache/lvmcache). That’s probably better to control policy and has no rebuild penalty.
– eckes
35 mins ago










1 Answer
1






active

oldest

votes

















up vote
3
down vote













Hmm, I'm not sure I would want to mix two so different volumes in a single RAID1. If you do that half of your requests will be served from the slower EBS and half from the faster instance storage and that may lead to quite an unpredictable performance. I would look at standard tools to achieve a better performance.



Look at Provisioned IOPS EBS disks (if you need high random-access IO) or Throughput optimised EBS (if you're sequentially reading large files). They may provide the performance you need out of the box. The pricing is here.



You should also look at some caching, especially as it's mostly read-only contents as you say. Every time the file is needed you can have a look in the local cache dir on the ephemeral storage and if it's there serve it from there. If not take it from EBS and save a copy in the cache. Especially if it's all read only it should be quite a simple caching layer.



Or if the files on EBS are database files (which I suspect may be the case) cache the results of your queries or processing in Memcache or Redis or in the database native cache (e.g. MySQL Query Cache).



Hope that helps :)






share|improve this answer





















  • That's what I'm testing - could be that the first read satisfies the request, so all the reads will be served from the faster disk.
    – Criggie
    2 hours ago










  • Aslo, IO1 disk with maximum IOPs is not as fast as ephemeral disk for this particular use case, plus IO1 is quite pricy at that level.
    – Criggie
    2 hours ago











Your Answer








StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "2"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f945590%2famazon-aws-ephemeral-disks-and-raid1%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
3
down vote













Hmm, I'm not sure I would want to mix two so different volumes in a single RAID1. If you do that half of your requests will be served from the slower EBS and half from the faster instance storage and that may lead to quite an unpredictable performance. I would look at standard tools to achieve a better performance.



Look at Provisioned IOPS EBS disks (if you need high random-access IO) or Throughput optimised EBS (if you're sequentially reading large files). They may provide the performance you need out of the box. The pricing is here.



You should also look at some caching, especially as it's mostly read-only contents as you say. Every time the file is needed you can have a look in the local cache dir on the ephemeral storage and if it's there serve it from there. If not take it from EBS and save a copy in the cache. Especially if it's all read only it should be quite a simple caching layer.



Or if the files on EBS are database files (which I suspect may be the case) cache the results of your queries or processing in Memcache or Redis or in the database native cache (e.g. MySQL Query Cache).



Hope that helps :)






share|improve this answer





















  • That's what I'm testing - could be that the first read satisfies the request, so all the reads will be served from the faster disk.
    – Criggie
    2 hours ago










  • Aslo, IO1 disk with maximum IOPs is not as fast as ephemeral disk for this particular use case, plus IO1 is quite pricy at that level.
    – Criggie
    2 hours ago















up vote
3
down vote













Hmm, I'm not sure I would want to mix two so different volumes in a single RAID1. If you do that half of your requests will be served from the slower EBS and half from the faster instance storage and that may lead to quite an unpredictable performance. I would look at standard tools to achieve a better performance.



Look at Provisioned IOPS EBS disks (if you need high random-access IO) or Throughput optimised EBS (if you're sequentially reading large files). They may provide the performance you need out of the box. The pricing is here.



You should also look at some caching, especially as it's mostly read-only contents as you say. Every time the file is needed you can have a look in the local cache dir on the ephemeral storage and if it's there serve it from there. If not take it from EBS and save a copy in the cache. Especially if it's all read only it should be quite a simple caching layer.



Or if the files on EBS are database files (which I suspect may be the case) cache the results of your queries or processing in Memcache or Redis or in the database native cache (e.g. MySQL Query Cache).



Hope that helps :)






share|improve this answer





















  • That's what I'm testing - could be that the first read satisfies the request, so all the reads will be served from the faster disk.
    – Criggie
    2 hours ago










  • Aslo, IO1 disk with maximum IOPs is not as fast as ephemeral disk for this particular use case, plus IO1 is quite pricy at that level.
    – Criggie
    2 hours ago













up vote
3
down vote










up vote
3
down vote









Hmm, I'm not sure I would want to mix two so different volumes in a single RAID1. If you do that half of your requests will be served from the slower EBS and half from the faster instance storage and that may lead to quite an unpredictable performance. I would look at standard tools to achieve a better performance.



Look at Provisioned IOPS EBS disks (if you need high random-access IO) or Throughput optimised EBS (if you're sequentially reading large files). They may provide the performance you need out of the box. The pricing is here.



You should also look at some caching, especially as it's mostly read-only contents as you say. Every time the file is needed you can have a look in the local cache dir on the ephemeral storage and if it's there serve it from there. If not take it from EBS and save a copy in the cache. Especially if it's all read only it should be quite a simple caching layer.



Or if the files on EBS are database files (which I suspect may be the case) cache the results of your queries or processing in Memcache or Redis or in the database native cache (e.g. MySQL Query Cache).



Hope that helps :)






share|improve this answer












Hmm, I'm not sure I would want to mix two so different volumes in a single RAID1. If you do that half of your requests will be served from the slower EBS and half from the faster instance storage and that may lead to quite an unpredictable performance. I would look at standard tools to achieve a better performance.



Look at Provisioned IOPS EBS disks (if you need high random-access IO) or Throughput optimised EBS (if you're sequentially reading large files). They may provide the performance you need out of the box. The pricing is here.



You should also look at some caching, especially as it's mostly read-only contents as you say. Every time the file is needed you can have a look in the local cache dir on the ephemeral storage and if it's there serve it from there. If not take it from EBS and save a copy in the cache. Especially if it's all read only it should be quite a simple caching layer.



Or if the files on EBS are database files (which I suspect may be the case) cache the results of your queries or processing in Memcache or Redis or in the database native cache (e.g. MySQL Query Cache).



Hope that helps :)







share|improve this answer












share|improve this answer



share|improve this answer










answered 3 hours ago









MLu

5,84911637




5,84911637












  • That's what I'm testing - could be that the first read satisfies the request, so all the reads will be served from the faster disk.
    – Criggie
    2 hours ago










  • Aslo, IO1 disk with maximum IOPs is not as fast as ephemeral disk for this particular use case, plus IO1 is quite pricy at that level.
    – Criggie
    2 hours ago


















  • That's what I'm testing - could be that the first read satisfies the request, so all the reads will be served from the faster disk.
    – Criggie
    2 hours ago










  • Aslo, IO1 disk with maximum IOPs is not as fast as ephemeral disk for this particular use case, plus IO1 is quite pricy at that level.
    – Criggie
    2 hours ago
















That's what I'm testing - could be that the first read satisfies the request, so all the reads will be served from the faster disk.
– Criggie
2 hours ago




That's what I'm testing - could be that the first read satisfies the request, so all the reads will be served from the faster disk.
– Criggie
2 hours ago












Aslo, IO1 disk with maximum IOPs is not as fast as ephemeral disk for this particular use case, plus IO1 is quite pricy at that level.
– Criggie
2 hours ago




Aslo, IO1 disk with maximum IOPs is not as fast as ephemeral disk for this particular use case, plus IO1 is quite pricy at that level.
– Criggie
2 hours ago


















draft saved

draft discarded




















































Thanks for contributing an answer to Server Fault!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f945590%2famazon-aws-ephemeral-disks-and-raid1%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Ellipse (mathématiques)

Quarter-circle Tiles

Mont Emei