A function that generates arrays of video thumbnails using canvas











up vote
1
down vote

favorite












I have just finished my first version of a function that returns an array of video screenshots created with canvas. What I am looking for are ways to improve this speed by as much as possible. I want to almost be able to instantly display the results. Yes, this should be a backend thing, no I can't do it in the backend. It needs to be done in the frontend.



One of my biggest performance issues at the moment, or what I at least think it is, is the fact that everytime a new image is pushed inside the array I call the callback function. This is NOT the desired functionality, I wish to return the contents of the result only when the images array is the same length as the given amount parameter. However, this is outside of my ability because I simply do not know how this could be accomplished. If anyone knows a way to fix this, please elaborate.



One more issue I am having is the fact that my renderImage function keeps old images from previous videos. I have no idea why or how I can fix this. Help is appreciated but my biggest priority is to increase the render and display speed of the fake generated gif's (src loops in this case).






// Max Screenshots (Firefox) = 19
// Max Screenshots (Chrome) = Infinite

const generateVideoThumbnail = (
event,
callback,
options = {
amount: 15,
step: 2
},
) => {
let {
amount,
step
} = options;
if (amount > 19) {
amount = 19;
}
if (step > 4) {
step = 4;
}

const fileReader = new FileReader();
const response = {
status: '',
images: ,
firstImage: null
};
const videoBlob = event.target.files[0];

if (videoBlob.type.match('video')) {
fileReader.onload = () => {
const blob = new Blob([fileReader.result], {
type: videoBlob.type
});
const url = URL.createObjectURL(blob);
const video = document.createElement('video');

const snapImage = () => {
const canvas = document.createElement('canvas');
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
canvas
.getContext('2d')
.drawImage(video, 0, 0, canvas.width, canvas.height);

const image = canvas.toDataURL();
const success = image.length > 100000;

if (success) {
response.status = 'Video successfully parsed.';
response.images.push(image);

if (!response.firstImage) {
response.firstImage = image;
}

video.currentTime += step;
URL.revokeObjectURL(url);
callback(response);
}

return success;
};

const timeupdate = () => {
snapImage();
if (response.images.length < amount) {
snapImage();
} else {
console.log(response);
video.removeEventListener('timeupdate', timeupdate);
video.pause();
}
};

video.addEventListener('loadeddata', () => {
snapImage();
});

video.addEventListener('timeupdate', timeupdate);
video.preload = 'metadata';
video.src = url;
video.muted = true;
video.playsInline = true;
video.play();
};
} else {
response.status = 'File is not a video.';
response.images = null;
return callback(response);
}

fileReader.readAsArrayBuffer(videoBlob);
};

const renderImage = (res, amount) => {
const gif = document.getElementById('gif');

let imageIndex = 0;
let timer = undefined;

const gifAnimation = () => {
gif.src = res.images[imageIndex];
imageIndex++;

if (imageIndex === res.images.length - 1) {
imageIndex = 0;
}

timer = setTimeout(gifAnimation, 225);
};

const mouseover = () => {
gifAnimation();
};

const mouseleave = () => {
clearTimeout(timer);
gif.src = res.firstImage;
imageIndex = 0;
};

gif.removeEventListener('mouseover', mouseover);
gif.removeEventListener('mouseleave', mouseleave);

if (res.images.length === amount) {
gif.src = res.firstImage;

gif.addEventListener('mouseover', mouseover);
gif.addEventListener('mouseleave', mouseleave);
}
};

document.getElementById('inputFile').addEventListener('change', event =>
generateVideoThumbnail(event, res => renderImage(res, 15), {
amount: 15,
step: 2,
}),
);

<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="X-UA-Compatible" content="ie=edge" />
<title>Demo 3</title>

<style>
body {
margin: 0;
padding: 0;
text-align: center;
}

img {
display: block;
height: 33vh;
object-fit: cover;
transition: all .2s ease-in-out;
width: auto;
}

#gif {
margin: 5% auto;
}
</style>
</head>

<body>
<img id="gif" />
<input type="file" id="inputFile" accept=".mov, .mp4" />
<script src="./index.js"></script>
</body>

</html>












share|improve this question






















  • There are syntax errors at stacksnippets
    – guest271314
    20 hours ago












  • What is the purpose of video.currentTime += step?
    – guest271314
    20 hours ago















up vote
1
down vote

favorite












I have just finished my first version of a function that returns an array of video screenshots created with canvas. What I am looking for are ways to improve this speed by as much as possible. I want to almost be able to instantly display the results. Yes, this should be a backend thing, no I can't do it in the backend. It needs to be done in the frontend.



One of my biggest performance issues at the moment, or what I at least think it is, is the fact that everytime a new image is pushed inside the array I call the callback function. This is NOT the desired functionality, I wish to return the contents of the result only when the images array is the same length as the given amount parameter. However, this is outside of my ability because I simply do not know how this could be accomplished. If anyone knows a way to fix this, please elaborate.



One more issue I am having is the fact that my renderImage function keeps old images from previous videos. I have no idea why or how I can fix this. Help is appreciated but my biggest priority is to increase the render and display speed of the fake generated gif's (src loops in this case).






// Max Screenshots (Firefox) = 19
// Max Screenshots (Chrome) = Infinite

const generateVideoThumbnail = (
event,
callback,
options = {
amount: 15,
step: 2
},
) => {
let {
amount,
step
} = options;
if (amount > 19) {
amount = 19;
}
if (step > 4) {
step = 4;
}

const fileReader = new FileReader();
const response = {
status: '',
images: ,
firstImage: null
};
const videoBlob = event.target.files[0];

if (videoBlob.type.match('video')) {
fileReader.onload = () => {
const blob = new Blob([fileReader.result], {
type: videoBlob.type
});
const url = URL.createObjectURL(blob);
const video = document.createElement('video');

const snapImage = () => {
const canvas = document.createElement('canvas');
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
canvas
.getContext('2d')
.drawImage(video, 0, 0, canvas.width, canvas.height);

const image = canvas.toDataURL();
const success = image.length > 100000;

if (success) {
response.status = 'Video successfully parsed.';
response.images.push(image);

if (!response.firstImage) {
response.firstImage = image;
}

video.currentTime += step;
URL.revokeObjectURL(url);
callback(response);
}

return success;
};

const timeupdate = () => {
snapImage();
if (response.images.length < amount) {
snapImage();
} else {
console.log(response);
video.removeEventListener('timeupdate', timeupdate);
video.pause();
}
};

video.addEventListener('loadeddata', () => {
snapImage();
});

video.addEventListener('timeupdate', timeupdate);
video.preload = 'metadata';
video.src = url;
video.muted = true;
video.playsInline = true;
video.play();
};
} else {
response.status = 'File is not a video.';
response.images = null;
return callback(response);
}

fileReader.readAsArrayBuffer(videoBlob);
};

const renderImage = (res, amount) => {
const gif = document.getElementById('gif');

let imageIndex = 0;
let timer = undefined;

const gifAnimation = () => {
gif.src = res.images[imageIndex];
imageIndex++;

if (imageIndex === res.images.length - 1) {
imageIndex = 0;
}

timer = setTimeout(gifAnimation, 225);
};

const mouseover = () => {
gifAnimation();
};

const mouseleave = () => {
clearTimeout(timer);
gif.src = res.firstImage;
imageIndex = 0;
};

gif.removeEventListener('mouseover', mouseover);
gif.removeEventListener('mouseleave', mouseleave);

if (res.images.length === amount) {
gif.src = res.firstImage;

gif.addEventListener('mouseover', mouseover);
gif.addEventListener('mouseleave', mouseleave);
}
};

document.getElementById('inputFile').addEventListener('change', event =>
generateVideoThumbnail(event, res => renderImage(res, 15), {
amount: 15,
step: 2,
}),
);

<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="X-UA-Compatible" content="ie=edge" />
<title>Demo 3</title>

<style>
body {
margin: 0;
padding: 0;
text-align: center;
}

img {
display: block;
height: 33vh;
object-fit: cover;
transition: all .2s ease-in-out;
width: auto;
}

#gif {
margin: 5% auto;
}
</style>
</head>

<body>
<img id="gif" />
<input type="file" id="inputFile" accept=".mov, .mp4" />
<script src="./index.js"></script>
</body>

</html>












share|improve this question






















  • There are syntax errors at stacksnippets
    – guest271314
    20 hours ago












  • What is the purpose of video.currentTime += step?
    – guest271314
    20 hours ago













up vote
1
down vote

favorite









up vote
1
down vote

favorite











I have just finished my first version of a function that returns an array of video screenshots created with canvas. What I am looking for are ways to improve this speed by as much as possible. I want to almost be able to instantly display the results. Yes, this should be a backend thing, no I can't do it in the backend. It needs to be done in the frontend.



One of my biggest performance issues at the moment, or what I at least think it is, is the fact that everytime a new image is pushed inside the array I call the callback function. This is NOT the desired functionality, I wish to return the contents of the result only when the images array is the same length as the given amount parameter. However, this is outside of my ability because I simply do not know how this could be accomplished. If anyone knows a way to fix this, please elaborate.



One more issue I am having is the fact that my renderImage function keeps old images from previous videos. I have no idea why or how I can fix this. Help is appreciated but my biggest priority is to increase the render and display speed of the fake generated gif's (src loops in this case).






// Max Screenshots (Firefox) = 19
// Max Screenshots (Chrome) = Infinite

const generateVideoThumbnail = (
event,
callback,
options = {
amount: 15,
step: 2
},
) => {
let {
amount,
step
} = options;
if (amount > 19) {
amount = 19;
}
if (step > 4) {
step = 4;
}

const fileReader = new FileReader();
const response = {
status: '',
images: ,
firstImage: null
};
const videoBlob = event.target.files[0];

if (videoBlob.type.match('video')) {
fileReader.onload = () => {
const blob = new Blob([fileReader.result], {
type: videoBlob.type
});
const url = URL.createObjectURL(blob);
const video = document.createElement('video');

const snapImage = () => {
const canvas = document.createElement('canvas');
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
canvas
.getContext('2d')
.drawImage(video, 0, 0, canvas.width, canvas.height);

const image = canvas.toDataURL();
const success = image.length > 100000;

if (success) {
response.status = 'Video successfully parsed.';
response.images.push(image);

if (!response.firstImage) {
response.firstImage = image;
}

video.currentTime += step;
URL.revokeObjectURL(url);
callback(response);
}

return success;
};

const timeupdate = () => {
snapImage();
if (response.images.length < amount) {
snapImage();
} else {
console.log(response);
video.removeEventListener('timeupdate', timeupdate);
video.pause();
}
};

video.addEventListener('loadeddata', () => {
snapImage();
});

video.addEventListener('timeupdate', timeupdate);
video.preload = 'metadata';
video.src = url;
video.muted = true;
video.playsInline = true;
video.play();
};
} else {
response.status = 'File is not a video.';
response.images = null;
return callback(response);
}

fileReader.readAsArrayBuffer(videoBlob);
};

const renderImage = (res, amount) => {
const gif = document.getElementById('gif');

let imageIndex = 0;
let timer = undefined;

const gifAnimation = () => {
gif.src = res.images[imageIndex];
imageIndex++;

if (imageIndex === res.images.length - 1) {
imageIndex = 0;
}

timer = setTimeout(gifAnimation, 225);
};

const mouseover = () => {
gifAnimation();
};

const mouseleave = () => {
clearTimeout(timer);
gif.src = res.firstImage;
imageIndex = 0;
};

gif.removeEventListener('mouseover', mouseover);
gif.removeEventListener('mouseleave', mouseleave);

if (res.images.length === amount) {
gif.src = res.firstImage;

gif.addEventListener('mouseover', mouseover);
gif.addEventListener('mouseleave', mouseleave);
}
};

document.getElementById('inputFile').addEventListener('change', event =>
generateVideoThumbnail(event, res => renderImage(res, 15), {
amount: 15,
step: 2,
}),
);

<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="X-UA-Compatible" content="ie=edge" />
<title>Demo 3</title>

<style>
body {
margin: 0;
padding: 0;
text-align: center;
}

img {
display: block;
height: 33vh;
object-fit: cover;
transition: all .2s ease-in-out;
width: auto;
}

#gif {
margin: 5% auto;
}
</style>
</head>

<body>
<img id="gif" />
<input type="file" id="inputFile" accept=".mov, .mp4" />
<script src="./index.js"></script>
</body>

</html>












share|improve this question













I have just finished my first version of a function that returns an array of video screenshots created with canvas. What I am looking for are ways to improve this speed by as much as possible. I want to almost be able to instantly display the results. Yes, this should be a backend thing, no I can't do it in the backend. It needs to be done in the frontend.



One of my biggest performance issues at the moment, or what I at least think it is, is the fact that everytime a new image is pushed inside the array I call the callback function. This is NOT the desired functionality, I wish to return the contents of the result only when the images array is the same length as the given amount parameter. However, this is outside of my ability because I simply do not know how this could be accomplished. If anyone knows a way to fix this, please elaborate.



One more issue I am having is the fact that my renderImage function keeps old images from previous videos. I have no idea why or how I can fix this. Help is appreciated but my biggest priority is to increase the render and display speed of the fake generated gif's (src loops in this case).






// Max Screenshots (Firefox) = 19
// Max Screenshots (Chrome) = Infinite

const generateVideoThumbnail = (
event,
callback,
options = {
amount: 15,
step: 2
},
) => {
let {
amount,
step
} = options;
if (amount > 19) {
amount = 19;
}
if (step > 4) {
step = 4;
}

const fileReader = new FileReader();
const response = {
status: '',
images: ,
firstImage: null
};
const videoBlob = event.target.files[0];

if (videoBlob.type.match('video')) {
fileReader.onload = () => {
const blob = new Blob([fileReader.result], {
type: videoBlob.type
});
const url = URL.createObjectURL(blob);
const video = document.createElement('video');

const snapImage = () => {
const canvas = document.createElement('canvas');
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
canvas
.getContext('2d')
.drawImage(video, 0, 0, canvas.width, canvas.height);

const image = canvas.toDataURL();
const success = image.length > 100000;

if (success) {
response.status = 'Video successfully parsed.';
response.images.push(image);

if (!response.firstImage) {
response.firstImage = image;
}

video.currentTime += step;
URL.revokeObjectURL(url);
callback(response);
}

return success;
};

const timeupdate = () => {
snapImage();
if (response.images.length < amount) {
snapImage();
} else {
console.log(response);
video.removeEventListener('timeupdate', timeupdate);
video.pause();
}
};

video.addEventListener('loadeddata', () => {
snapImage();
});

video.addEventListener('timeupdate', timeupdate);
video.preload = 'metadata';
video.src = url;
video.muted = true;
video.playsInline = true;
video.play();
};
} else {
response.status = 'File is not a video.';
response.images = null;
return callback(response);
}

fileReader.readAsArrayBuffer(videoBlob);
};

const renderImage = (res, amount) => {
const gif = document.getElementById('gif');

let imageIndex = 0;
let timer = undefined;

const gifAnimation = () => {
gif.src = res.images[imageIndex];
imageIndex++;

if (imageIndex === res.images.length - 1) {
imageIndex = 0;
}

timer = setTimeout(gifAnimation, 225);
};

const mouseover = () => {
gifAnimation();
};

const mouseleave = () => {
clearTimeout(timer);
gif.src = res.firstImage;
imageIndex = 0;
};

gif.removeEventListener('mouseover', mouseover);
gif.removeEventListener('mouseleave', mouseleave);

if (res.images.length === amount) {
gif.src = res.firstImage;

gif.addEventListener('mouseover', mouseover);
gif.addEventListener('mouseleave', mouseleave);
}
};

document.getElementById('inputFile').addEventListener('change', event =>
generateVideoThumbnail(event, res => renderImage(res, 15), {
amount: 15,
step: 2,
}),
);

<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="X-UA-Compatible" content="ie=edge" />
<title>Demo 3</title>

<style>
body {
margin: 0;
padding: 0;
text-align: center;
}

img {
display: block;
height: 33vh;
object-fit: cover;
transition: all .2s ease-in-out;
width: auto;
}

#gif {
margin: 5% auto;
}
</style>
</head>

<body>
<img id="gif" />
<input type="file" id="inputFile" accept=".mov, .mp4" />
<script src="./index.js"></script>
</body>

</html>








// Max Screenshots (Firefox) = 19
// Max Screenshots (Chrome) = Infinite

const generateVideoThumbnail = (
event,
callback,
options = {
amount: 15,
step: 2
},
) => {
let {
amount,
step
} = options;
if (amount > 19) {
amount = 19;
}
if (step > 4) {
step = 4;
}

const fileReader = new FileReader();
const response = {
status: '',
images: ,
firstImage: null
};
const videoBlob = event.target.files[0];

if (videoBlob.type.match('video')) {
fileReader.onload = () => {
const blob = new Blob([fileReader.result], {
type: videoBlob.type
});
const url = URL.createObjectURL(blob);
const video = document.createElement('video');

const snapImage = () => {
const canvas = document.createElement('canvas');
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
canvas
.getContext('2d')
.drawImage(video, 0, 0, canvas.width, canvas.height);

const image = canvas.toDataURL();
const success = image.length > 100000;

if (success) {
response.status = 'Video successfully parsed.';
response.images.push(image);

if (!response.firstImage) {
response.firstImage = image;
}

video.currentTime += step;
URL.revokeObjectURL(url);
callback(response);
}

return success;
};

const timeupdate = () => {
snapImage();
if (response.images.length < amount) {
snapImage();
} else {
console.log(response);
video.removeEventListener('timeupdate', timeupdate);
video.pause();
}
};

video.addEventListener('loadeddata', () => {
snapImage();
});

video.addEventListener('timeupdate', timeupdate);
video.preload = 'metadata';
video.src = url;
video.muted = true;
video.playsInline = true;
video.play();
};
} else {
response.status = 'File is not a video.';
response.images = null;
return callback(response);
}

fileReader.readAsArrayBuffer(videoBlob);
};

const renderImage = (res, amount) => {
const gif = document.getElementById('gif');

let imageIndex = 0;
let timer = undefined;

const gifAnimation = () => {
gif.src = res.images[imageIndex];
imageIndex++;

if (imageIndex === res.images.length - 1) {
imageIndex = 0;
}

timer = setTimeout(gifAnimation, 225);
};

const mouseover = () => {
gifAnimation();
};

const mouseleave = () => {
clearTimeout(timer);
gif.src = res.firstImage;
imageIndex = 0;
};

gif.removeEventListener('mouseover', mouseover);
gif.removeEventListener('mouseleave', mouseleave);

if (res.images.length === amount) {
gif.src = res.firstImage;

gif.addEventListener('mouseover', mouseover);
gif.addEventListener('mouseleave', mouseleave);
}
};

document.getElementById('inputFile').addEventListener('change', event =>
generateVideoThumbnail(event, res => renderImage(res, 15), {
amount: 15,
step: 2,
}),
);

<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="X-UA-Compatible" content="ie=edge" />
<title>Demo 3</title>

<style>
body {
margin: 0;
padding: 0;
text-align: center;
}

img {
display: block;
height: 33vh;
object-fit: cover;
transition: all .2s ease-in-out;
width: auto;
}

#gif {
margin: 5% auto;
}
</style>
</head>

<body>
<img id="gif" />
<input type="file" id="inputFile" accept=".mov, .mp4" />
<script src="./index.js"></script>
</body>

</html>





// Max Screenshots (Firefox) = 19
// Max Screenshots (Chrome) = Infinite

const generateVideoThumbnail = (
event,
callback,
options = {
amount: 15,
step: 2
},
) => {
let {
amount,
step
} = options;
if (amount > 19) {
amount = 19;
}
if (step > 4) {
step = 4;
}

const fileReader = new FileReader();
const response = {
status: '',
images: ,
firstImage: null
};
const videoBlob = event.target.files[0];

if (videoBlob.type.match('video')) {
fileReader.onload = () => {
const blob = new Blob([fileReader.result], {
type: videoBlob.type
});
const url = URL.createObjectURL(blob);
const video = document.createElement('video');

const snapImage = () => {
const canvas = document.createElement('canvas');
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
canvas
.getContext('2d')
.drawImage(video, 0, 0, canvas.width, canvas.height);

const image = canvas.toDataURL();
const success = image.length > 100000;

if (success) {
response.status = 'Video successfully parsed.';
response.images.push(image);

if (!response.firstImage) {
response.firstImage = image;
}

video.currentTime += step;
URL.revokeObjectURL(url);
callback(response);
}

return success;
};

const timeupdate = () => {
snapImage();
if (response.images.length < amount) {
snapImage();
} else {
console.log(response);
video.removeEventListener('timeupdate', timeupdate);
video.pause();
}
};

video.addEventListener('loadeddata', () => {
snapImage();
});

video.addEventListener('timeupdate', timeupdate);
video.preload = 'metadata';
video.src = url;
video.muted = true;
video.playsInline = true;
video.play();
};
} else {
response.status = 'File is not a video.';
response.images = null;
return callback(response);
}

fileReader.readAsArrayBuffer(videoBlob);
};

const renderImage = (res, amount) => {
const gif = document.getElementById('gif');

let imageIndex = 0;
let timer = undefined;

const gifAnimation = () => {
gif.src = res.images[imageIndex];
imageIndex++;

if (imageIndex === res.images.length - 1) {
imageIndex = 0;
}

timer = setTimeout(gifAnimation, 225);
};

const mouseover = () => {
gifAnimation();
};

const mouseleave = () => {
clearTimeout(timer);
gif.src = res.firstImage;
imageIndex = 0;
};

gif.removeEventListener('mouseover', mouseover);
gif.removeEventListener('mouseleave', mouseleave);

if (res.images.length === amount) {
gif.src = res.firstImage;

gif.addEventListener('mouseover', mouseover);
gif.addEventListener('mouseleave', mouseleave);
}
};

document.getElementById('inputFile').addEventListener('change', event =>
generateVideoThumbnail(event, res => renderImage(res, 15), {
amount: 15,
step: 2,
}),
);

<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="X-UA-Compatible" content="ie=edge" />
<title>Demo 3</title>

<style>
body {
margin: 0;
padding: 0;
text-align: center;
}

img {
display: block;
height: 33vh;
object-fit: cover;
transition: all .2s ease-in-out;
width: auto;
}

#gif {
margin: 5% auto;
}
</style>
</head>

<body>
<img id="gif" />
<input type="file" id="inputFile" accept=".mov, .mp4" />
<script src="./index.js"></script>
</body>

</html>






javascript html5 canvas video






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 16 at 13:36









Vera Perrone

1688




1688












  • There are syntax errors at stacksnippets
    – guest271314
    20 hours ago












  • What is the purpose of video.currentTime += step?
    – guest271314
    20 hours ago


















  • There are syntax errors at stacksnippets
    – guest271314
    20 hours ago












  • What is the purpose of video.currentTime += step?
    – guest271314
    20 hours ago
















There are syntax errors at stacksnippets
– guest271314
20 hours ago






There are syntax errors at stacksnippets
– guest271314
20 hours ago














What is the purpose of video.currentTime += step?
– guest271314
20 hours ago




What is the purpose of video.currentTime += step?
– guest271314
20 hours ago










2 Answers
2






active

oldest

votes

















up vote
2
down vote













Wrong approch



Data URL's are for transport only.



A data URL converts a binary stream to a format that can be transported over networks that handle text files as UTF-8. It does this by EXPANDING the data to fit a safe range of characters (64 of them) storing 6 bits in every byte (UTF-8). That means for every 3 bytes in the source the data URL needs 4 bytes to store.



This is compounded in javascript by the fact that JavaScript strings are 16bit UCS-2 characters. That means that for every 3 bytes in the source the JavaScript dataURL needs 8 bytes to store.



Video



Videos are highly compressed streams, not only is each frame compressed, but that compression is related to previous frames. This allows very high compression rates. For example I picked a random low res video 420 by 270 at 20Fps, running time of 1h 25m, and a file size of 380MB.



A random frame saved as png was 238kb in size.



The data throughput to display is about 10mb a second (and this is low res LOW quality video, full frame HD is about 0.5Gig a second)



That means that running your function on that video the memory needed to complete the function is about 380 * 1024 * 1024 + (1 * 60 * 60 * 20fps + 25 * 60 * 20fps) * 200kb * 1024 * ((8 / 3)dataUrl Storage overhead) which is around 56 gigabytes of RAM (Something only the very top range devices can handle).



If you compare the Video size to the image dataURLs used to store the frames you have expanded the video memory needs to almost 400 times the original size.



Even if you consider better options, store frames as Jpeg Blobs (near one to one of bin size) set at very low compression (eg jpeg quality 0.4) each frame requires ~12kb. The final RAM need to store the frames is (1 * 60 * 60 * 20fps + 25 * 60 * 20fps) * 12kb * 1024 = 1.2Gig (do-able on PC's and laptops if you don't count the IMG element overhead)



Though it will fit memory the processing will no less require a lot of time to render, compress, and store.



The Sad truth



Until browsers come with a way to use a disk based cache for large data needs, video editing like applications will not be able to provide fast random access frame editing like traditional native video editing apps.



What to do?



I did notice that there was a image element id named gif which hints that the application may be a video to gif like thing.



Again this will be slow, but it is do-able if you compress the video frames to gif as you go. Get video frame, quantize color, compress to GIF, delete frame. Use webWorkers to pipeline the quantize, and compress steps. Ensure that unsaved gif data size does not crash the page.



You should also consider options for resolution and frame rate reduction. Also add warnings for long (large) videos. It will be slow, there is nothing you can do, that is the nature of videos,






share|improve this answer





















  • Hey, first of all thank you a lot for the very detailed explanation. However, I do think I need to elaborate a bit more. Users can upload a maximum of 10mb, and most videos won’t be longer than 15 seconds. (Only Instagram Videos and Stories). With means that memory will never be a big issue I hope. What I want to do is get an an array with a maximum of x pictures with x seconds between them. This is actually going to be a uploaded video preview. Where users can hover there preview and I will loop trough the array to fake a gif animation. This is only to give users feedback.
    – Vera Perrone
    Nov 16 at 16:47










  • The same effect you get when you hover a YouTube video, but I need to do this client side. The server will handle everything else and once the upload progress is done, this will be deleted from memory. It is just a feedback mechanism that shows users what they just uploaded.
    – Vera Perrone
    Nov 16 at 16:49










  • @VeraPerrone videos can play way before they have fully loaded, thus getting frame preview client side will always take longer than to start watching the video. If you have many videos and want to show preview frames for each you would need to load each video completely, something the client will not appreciate. YouTube previews are stored as a single low quality jpeg image containing frames. It is the only way to preview client side. Loading the video to capture frames for a preview, you may as well just play the video directly as a preview and save the bother and memory.
    – Blindman67
    Nov 16 at 17:16


















up vote
0
down vote













requestAnimationFrame could be substituted for setTimeout and timeupdate event (which is not executed at a consistent rate) where necessary in the existing code. The same <canvas> element can be used and captured within the snap function.






share|improve this answer























    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["\$", "\$"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "196"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














     

    draft saved


    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f207809%2fa-function-that-generates-arrays-of-video-thumbnails-using-canvas%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    2
    down vote













    Wrong approch



    Data URL's are for transport only.



    A data URL converts a binary stream to a format that can be transported over networks that handle text files as UTF-8. It does this by EXPANDING the data to fit a safe range of characters (64 of them) storing 6 bits in every byte (UTF-8). That means for every 3 bytes in the source the data URL needs 4 bytes to store.



    This is compounded in javascript by the fact that JavaScript strings are 16bit UCS-2 characters. That means that for every 3 bytes in the source the JavaScript dataURL needs 8 bytes to store.



    Video



    Videos are highly compressed streams, not only is each frame compressed, but that compression is related to previous frames. This allows very high compression rates. For example I picked a random low res video 420 by 270 at 20Fps, running time of 1h 25m, and a file size of 380MB.



    A random frame saved as png was 238kb in size.



    The data throughput to display is about 10mb a second (and this is low res LOW quality video, full frame HD is about 0.5Gig a second)



    That means that running your function on that video the memory needed to complete the function is about 380 * 1024 * 1024 + (1 * 60 * 60 * 20fps + 25 * 60 * 20fps) * 200kb * 1024 * ((8 / 3)dataUrl Storage overhead) which is around 56 gigabytes of RAM (Something only the very top range devices can handle).



    If you compare the Video size to the image dataURLs used to store the frames you have expanded the video memory needs to almost 400 times the original size.



    Even if you consider better options, store frames as Jpeg Blobs (near one to one of bin size) set at very low compression (eg jpeg quality 0.4) each frame requires ~12kb. The final RAM need to store the frames is (1 * 60 * 60 * 20fps + 25 * 60 * 20fps) * 12kb * 1024 = 1.2Gig (do-able on PC's and laptops if you don't count the IMG element overhead)



    Though it will fit memory the processing will no less require a lot of time to render, compress, and store.



    The Sad truth



    Until browsers come with a way to use a disk based cache for large data needs, video editing like applications will not be able to provide fast random access frame editing like traditional native video editing apps.



    What to do?



    I did notice that there was a image element id named gif which hints that the application may be a video to gif like thing.



    Again this will be slow, but it is do-able if you compress the video frames to gif as you go. Get video frame, quantize color, compress to GIF, delete frame. Use webWorkers to pipeline the quantize, and compress steps. Ensure that unsaved gif data size does not crash the page.



    You should also consider options for resolution and frame rate reduction. Also add warnings for long (large) videos. It will be slow, there is nothing you can do, that is the nature of videos,






    share|improve this answer





















    • Hey, first of all thank you a lot for the very detailed explanation. However, I do think I need to elaborate a bit more. Users can upload a maximum of 10mb, and most videos won’t be longer than 15 seconds. (Only Instagram Videos and Stories). With means that memory will never be a big issue I hope. What I want to do is get an an array with a maximum of x pictures with x seconds between them. This is actually going to be a uploaded video preview. Where users can hover there preview and I will loop trough the array to fake a gif animation. This is only to give users feedback.
      – Vera Perrone
      Nov 16 at 16:47










    • The same effect you get when you hover a YouTube video, but I need to do this client side. The server will handle everything else and once the upload progress is done, this will be deleted from memory. It is just a feedback mechanism that shows users what they just uploaded.
      – Vera Perrone
      Nov 16 at 16:49










    • @VeraPerrone videos can play way before they have fully loaded, thus getting frame preview client side will always take longer than to start watching the video. If you have many videos and want to show preview frames for each you would need to load each video completely, something the client will not appreciate. YouTube previews are stored as a single low quality jpeg image containing frames. It is the only way to preview client side. Loading the video to capture frames for a preview, you may as well just play the video directly as a preview and save the bother and memory.
      – Blindman67
      Nov 16 at 17:16















    up vote
    2
    down vote













    Wrong approch



    Data URL's are for transport only.



    A data URL converts a binary stream to a format that can be transported over networks that handle text files as UTF-8. It does this by EXPANDING the data to fit a safe range of characters (64 of them) storing 6 bits in every byte (UTF-8). That means for every 3 bytes in the source the data URL needs 4 bytes to store.



    This is compounded in javascript by the fact that JavaScript strings are 16bit UCS-2 characters. That means that for every 3 bytes in the source the JavaScript dataURL needs 8 bytes to store.



    Video



    Videos are highly compressed streams, not only is each frame compressed, but that compression is related to previous frames. This allows very high compression rates. For example I picked a random low res video 420 by 270 at 20Fps, running time of 1h 25m, and a file size of 380MB.



    A random frame saved as png was 238kb in size.



    The data throughput to display is about 10mb a second (and this is low res LOW quality video, full frame HD is about 0.5Gig a second)



    That means that running your function on that video the memory needed to complete the function is about 380 * 1024 * 1024 + (1 * 60 * 60 * 20fps + 25 * 60 * 20fps) * 200kb * 1024 * ((8 / 3)dataUrl Storage overhead) which is around 56 gigabytes of RAM (Something only the very top range devices can handle).



    If you compare the Video size to the image dataURLs used to store the frames you have expanded the video memory needs to almost 400 times the original size.



    Even if you consider better options, store frames as Jpeg Blobs (near one to one of bin size) set at very low compression (eg jpeg quality 0.4) each frame requires ~12kb. The final RAM need to store the frames is (1 * 60 * 60 * 20fps + 25 * 60 * 20fps) * 12kb * 1024 = 1.2Gig (do-able on PC's and laptops if you don't count the IMG element overhead)



    Though it will fit memory the processing will no less require a lot of time to render, compress, and store.



    The Sad truth



    Until browsers come with a way to use a disk based cache for large data needs, video editing like applications will not be able to provide fast random access frame editing like traditional native video editing apps.



    What to do?



    I did notice that there was a image element id named gif which hints that the application may be a video to gif like thing.



    Again this will be slow, but it is do-able if you compress the video frames to gif as you go. Get video frame, quantize color, compress to GIF, delete frame. Use webWorkers to pipeline the quantize, and compress steps. Ensure that unsaved gif data size does not crash the page.



    You should also consider options for resolution and frame rate reduction. Also add warnings for long (large) videos. It will be slow, there is nothing you can do, that is the nature of videos,






    share|improve this answer





















    • Hey, first of all thank you a lot for the very detailed explanation. However, I do think I need to elaborate a bit more. Users can upload a maximum of 10mb, and most videos won’t be longer than 15 seconds. (Only Instagram Videos and Stories). With means that memory will never be a big issue I hope. What I want to do is get an an array with a maximum of x pictures with x seconds between them. This is actually going to be a uploaded video preview. Where users can hover there preview and I will loop trough the array to fake a gif animation. This is only to give users feedback.
      – Vera Perrone
      Nov 16 at 16:47










    • The same effect you get when you hover a YouTube video, but I need to do this client side. The server will handle everything else and once the upload progress is done, this will be deleted from memory. It is just a feedback mechanism that shows users what they just uploaded.
      – Vera Perrone
      Nov 16 at 16:49










    • @VeraPerrone videos can play way before they have fully loaded, thus getting frame preview client side will always take longer than to start watching the video. If you have many videos and want to show preview frames for each you would need to load each video completely, something the client will not appreciate. YouTube previews are stored as a single low quality jpeg image containing frames. It is the only way to preview client side. Loading the video to capture frames for a preview, you may as well just play the video directly as a preview and save the bother and memory.
      – Blindman67
      Nov 16 at 17:16













    up vote
    2
    down vote










    up vote
    2
    down vote









    Wrong approch



    Data URL's are for transport only.



    A data URL converts a binary stream to a format that can be transported over networks that handle text files as UTF-8. It does this by EXPANDING the data to fit a safe range of characters (64 of them) storing 6 bits in every byte (UTF-8). That means for every 3 bytes in the source the data URL needs 4 bytes to store.



    This is compounded in javascript by the fact that JavaScript strings are 16bit UCS-2 characters. That means that for every 3 bytes in the source the JavaScript dataURL needs 8 bytes to store.



    Video



    Videos are highly compressed streams, not only is each frame compressed, but that compression is related to previous frames. This allows very high compression rates. For example I picked a random low res video 420 by 270 at 20Fps, running time of 1h 25m, and a file size of 380MB.



    A random frame saved as png was 238kb in size.



    The data throughput to display is about 10mb a second (and this is low res LOW quality video, full frame HD is about 0.5Gig a second)



    That means that running your function on that video the memory needed to complete the function is about 380 * 1024 * 1024 + (1 * 60 * 60 * 20fps + 25 * 60 * 20fps) * 200kb * 1024 * ((8 / 3)dataUrl Storage overhead) which is around 56 gigabytes of RAM (Something only the very top range devices can handle).



    If you compare the Video size to the image dataURLs used to store the frames you have expanded the video memory needs to almost 400 times the original size.



    Even if you consider better options, store frames as Jpeg Blobs (near one to one of bin size) set at very low compression (eg jpeg quality 0.4) each frame requires ~12kb. The final RAM need to store the frames is (1 * 60 * 60 * 20fps + 25 * 60 * 20fps) * 12kb * 1024 = 1.2Gig (do-able on PC's and laptops if you don't count the IMG element overhead)



    Though it will fit memory the processing will no less require a lot of time to render, compress, and store.



    The Sad truth



    Until browsers come with a way to use a disk based cache for large data needs, video editing like applications will not be able to provide fast random access frame editing like traditional native video editing apps.



    What to do?



    I did notice that there was a image element id named gif which hints that the application may be a video to gif like thing.



    Again this will be slow, but it is do-able if you compress the video frames to gif as you go. Get video frame, quantize color, compress to GIF, delete frame. Use webWorkers to pipeline the quantize, and compress steps. Ensure that unsaved gif data size does not crash the page.



    You should also consider options for resolution and frame rate reduction. Also add warnings for long (large) videos. It will be slow, there is nothing you can do, that is the nature of videos,






    share|improve this answer












    Wrong approch



    Data URL's are for transport only.



    A data URL converts a binary stream to a format that can be transported over networks that handle text files as UTF-8. It does this by EXPANDING the data to fit a safe range of characters (64 of them) storing 6 bits in every byte (UTF-8). That means for every 3 bytes in the source the data URL needs 4 bytes to store.



    This is compounded in javascript by the fact that JavaScript strings are 16bit UCS-2 characters. That means that for every 3 bytes in the source the JavaScript dataURL needs 8 bytes to store.



    Video



    Videos are highly compressed streams, not only is each frame compressed, but that compression is related to previous frames. This allows very high compression rates. For example I picked a random low res video 420 by 270 at 20Fps, running time of 1h 25m, and a file size of 380MB.



    A random frame saved as png was 238kb in size.



    The data throughput to display is about 10mb a second (and this is low res LOW quality video, full frame HD is about 0.5Gig a second)



    That means that running your function on that video the memory needed to complete the function is about 380 * 1024 * 1024 + (1 * 60 * 60 * 20fps + 25 * 60 * 20fps) * 200kb * 1024 * ((8 / 3)dataUrl Storage overhead) which is around 56 gigabytes of RAM (Something only the very top range devices can handle).



    If you compare the Video size to the image dataURLs used to store the frames you have expanded the video memory needs to almost 400 times the original size.



    Even if you consider better options, store frames as Jpeg Blobs (near one to one of bin size) set at very low compression (eg jpeg quality 0.4) each frame requires ~12kb. The final RAM need to store the frames is (1 * 60 * 60 * 20fps + 25 * 60 * 20fps) * 12kb * 1024 = 1.2Gig (do-able on PC's and laptops if you don't count the IMG element overhead)



    Though it will fit memory the processing will no less require a lot of time to render, compress, and store.



    The Sad truth



    Until browsers come with a way to use a disk based cache for large data needs, video editing like applications will not be able to provide fast random access frame editing like traditional native video editing apps.



    What to do?



    I did notice that there was a image element id named gif which hints that the application may be a video to gif like thing.



    Again this will be slow, but it is do-able if you compress the video frames to gif as you go. Get video frame, quantize color, compress to GIF, delete frame. Use webWorkers to pipeline the quantize, and compress steps. Ensure that unsaved gif data size does not crash the page.



    You should also consider options for resolution and frame rate reduction. Also add warnings for long (large) videos. It will be slow, there is nothing you can do, that is the nature of videos,







    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Nov 16 at 15:39









    Blindman67

    6,4691521




    6,4691521












    • Hey, first of all thank you a lot for the very detailed explanation. However, I do think I need to elaborate a bit more. Users can upload a maximum of 10mb, and most videos won’t be longer than 15 seconds. (Only Instagram Videos and Stories). With means that memory will never be a big issue I hope. What I want to do is get an an array with a maximum of x pictures with x seconds between them. This is actually going to be a uploaded video preview. Where users can hover there preview and I will loop trough the array to fake a gif animation. This is only to give users feedback.
      – Vera Perrone
      Nov 16 at 16:47










    • The same effect you get when you hover a YouTube video, but I need to do this client side. The server will handle everything else and once the upload progress is done, this will be deleted from memory. It is just a feedback mechanism that shows users what they just uploaded.
      – Vera Perrone
      Nov 16 at 16:49










    • @VeraPerrone videos can play way before they have fully loaded, thus getting frame preview client side will always take longer than to start watching the video. If you have many videos and want to show preview frames for each you would need to load each video completely, something the client will not appreciate. YouTube previews are stored as a single low quality jpeg image containing frames. It is the only way to preview client side. Loading the video to capture frames for a preview, you may as well just play the video directly as a preview and save the bother and memory.
      – Blindman67
      Nov 16 at 17:16


















    • Hey, first of all thank you a lot for the very detailed explanation. However, I do think I need to elaborate a bit more. Users can upload a maximum of 10mb, and most videos won’t be longer than 15 seconds. (Only Instagram Videos and Stories). With means that memory will never be a big issue I hope. What I want to do is get an an array with a maximum of x pictures with x seconds between them. This is actually going to be a uploaded video preview. Where users can hover there preview and I will loop trough the array to fake a gif animation. This is only to give users feedback.
      – Vera Perrone
      Nov 16 at 16:47










    • The same effect you get when you hover a YouTube video, but I need to do this client side. The server will handle everything else and once the upload progress is done, this will be deleted from memory. It is just a feedback mechanism that shows users what they just uploaded.
      – Vera Perrone
      Nov 16 at 16:49










    • @VeraPerrone videos can play way before they have fully loaded, thus getting frame preview client side will always take longer than to start watching the video. If you have many videos and want to show preview frames for each you would need to load each video completely, something the client will not appreciate. YouTube previews are stored as a single low quality jpeg image containing frames. It is the only way to preview client side. Loading the video to capture frames for a preview, you may as well just play the video directly as a preview and save the bother and memory.
      – Blindman67
      Nov 16 at 17:16
















    Hey, first of all thank you a lot for the very detailed explanation. However, I do think I need to elaborate a bit more. Users can upload a maximum of 10mb, and most videos won’t be longer than 15 seconds. (Only Instagram Videos and Stories). With means that memory will never be a big issue I hope. What I want to do is get an an array with a maximum of x pictures with x seconds between them. This is actually going to be a uploaded video preview. Where users can hover there preview and I will loop trough the array to fake a gif animation. This is only to give users feedback.
    – Vera Perrone
    Nov 16 at 16:47




    Hey, first of all thank you a lot for the very detailed explanation. However, I do think I need to elaborate a bit more. Users can upload a maximum of 10mb, and most videos won’t be longer than 15 seconds. (Only Instagram Videos and Stories). With means that memory will never be a big issue I hope. What I want to do is get an an array with a maximum of x pictures with x seconds between them. This is actually going to be a uploaded video preview. Where users can hover there preview and I will loop trough the array to fake a gif animation. This is only to give users feedback.
    – Vera Perrone
    Nov 16 at 16:47












    The same effect you get when you hover a YouTube video, but I need to do this client side. The server will handle everything else and once the upload progress is done, this will be deleted from memory. It is just a feedback mechanism that shows users what they just uploaded.
    – Vera Perrone
    Nov 16 at 16:49




    The same effect you get when you hover a YouTube video, but I need to do this client side. The server will handle everything else and once the upload progress is done, this will be deleted from memory. It is just a feedback mechanism that shows users what they just uploaded.
    – Vera Perrone
    Nov 16 at 16:49












    @VeraPerrone videos can play way before they have fully loaded, thus getting frame preview client side will always take longer than to start watching the video. If you have many videos and want to show preview frames for each you would need to load each video completely, something the client will not appreciate. YouTube previews are stored as a single low quality jpeg image containing frames. It is the only way to preview client side. Loading the video to capture frames for a preview, you may as well just play the video directly as a preview and save the bother and memory.
    – Blindman67
    Nov 16 at 17:16




    @VeraPerrone videos can play way before they have fully loaded, thus getting frame preview client side will always take longer than to start watching the video. If you have many videos and want to show preview frames for each you would need to load each video completely, something the client will not appreciate. YouTube previews are stored as a single low quality jpeg image containing frames. It is the only way to preview client side. Loading the video to capture frames for a preview, you may as well just play the video directly as a preview and save the bother and memory.
    – Blindman67
    Nov 16 at 17:16












    up vote
    0
    down vote













    requestAnimationFrame could be substituted for setTimeout and timeupdate event (which is not executed at a consistent rate) where necessary in the existing code. The same <canvas> element can be used and captured within the snap function.






    share|improve this answer



























      up vote
      0
      down vote













      requestAnimationFrame could be substituted for setTimeout and timeupdate event (which is not executed at a consistent rate) where necessary in the existing code. The same <canvas> element can be used and captured within the snap function.






      share|improve this answer

























        up vote
        0
        down vote










        up vote
        0
        down vote









        requestAnimationFrame could be substituted for setTimeout and timeupdate event (which is not executed at a consistent rate) where necessary in the existing code. The same <canvas> element can be used and captured within the snap function.






        share|improve this answer














        requestAnimationFrame could be substituted for setTimeout and timeupdate event (which is not executed at a consistent rate) where necessary in the existing code. The same <canvas> element can be used and captured within the snap function.







        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited 20 hours ago

























        answered 21 hours ago









        guest271314

        1315




        1315






























             

            draft saved


            draft discarded



















































             


            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f207809%2fa-function-that-generates-arrays-of-video-thumbnails-using-canvas%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Ellipse (mathématiques)

            Quarter-circle Tiles

            Mont Emei