Jump to content
Search In
  • More options...
Find results that contain...
Find results in...

Deeplab AI image classification not working on node.js tensorflow

Go to solution Solved by TheNuzziNuzz,

I figured it out! Deeplab only works with version 1.3.1 of tensorflow. I downgraded my packages and it is working amazing. 

Tensorflow model deeplab: https://github.com/tensorflow/tfjs-models/tree/master/deeplab doesn't seem to work for me on node.js.

I haven't been able to find any example code for deeplab in nodejs. When I try to segment an image I get the error

error: The dtype of dict['ImageTensor'] provided in model.execute(dict) must be int32, but was float32

And googling the error seemed to turn up no helpful soloutions.

 

Would love thoughts from sombody who knows more!

Computers r fun

Link to post
Share on other sites
9 hours ago, TheNuzziNuzz said:

Tensorflow model deeplab: https://github.com/tensorflow/tfjs-models/tree/master/deeplab doesn't seem to work for me on node.js.

I haven't been able to find any example code for deeplab in nodejs. When I try to segment an image I get the error


error: The dtype of dict['ImageTensor'] provided in model.execute(dict) must be int32, but was float32

And googling the error seemed to turn up no helpful soloutions.

 

Would love thoughts from sombody who knows more!

Easy you're passing a float instead of an int. Also maybe consider using tensorflow lite with the pre-trained models https://www.tensorflow.org/lite/models/image_classification/overview

Link to post
Share on other sites
Just now, TheNuzziNuzz said:

They don't seem to have a nodejs api for lite. What is the benefit of using lite over regular?

Lighter to process. On that page you have a pre-trained model should work for what you're doing even if you're using regular tf.

Link to post
Share on other sites
2 minutes ago, TheNuzziNuzz said:

Neat thank you! How do I import these models into node? Do I just point tf.node.loadSavedModel to the tar.gz?

tf.loadGraphModel("https://tfhub.dev/google/tfjs-model/imagenet/inception_v1/classification/3/default/1", { fromTFHub: true })

 

Link to post
Share on other sites
6 minutes ago, zhnu said:

tf.loadGraphModel("https://tfhub.dev/google/tfjs-model/imagenet/inception_v1/classification/3/default/1", { fromTFHub: true })

 

How do I actually run the model? There is no documentation on the usage page. Thank you so much for helping me, I know this is such a noob situation, I'm not sure why i'm having such a hard time finding any tensorflow node.js documentation. I really appreciate your time.

Computers r fun

Link to post
Share on other sites
const predictions = await model.predict(tf.node.decodeImage(frameData))

results in

Error: The shape of dict['images'] provided in model.execute(dict) must be [-1,-1,-1,3], but was [480,720,3]

 

Computers r fun

Link to post
Share on other sites
9 minutes ago, TheNuzziNuzz said:

const predictions = await model.predict(tf.node.decodeImage(frameData))

results in


Error: The shape of dict['images'] provided in model.execute(dict) must be [-1,-1,-1,3], but was [480,720,3]

 

I think you really need to start with a tutorial before jumping right in:
https://developer.ibm.com/technologies/artificial-intelligence/tutorials/an-introduction-to-ai-in-nodejs/

Link to post
Share on other sites
12 minutes ago, zhnu said:

I think you really need to start with a tutorial before jumping right in:
https://developer.ibm.com/technologies/artificial-intelligence/tutorials/an-introduction-to-ai-in-nodejs/

I have gone through this, I understand it for the model in the example. However i'm not sure how that translates when using the models on TF hub. TF hub doesn't provide a link to the labels like is used in the example you cited.

Computers r fun

Link to post
Share on other sites
2 minutes ago, TheNuzziNuzz said:

I have gone through this, I understand it for the model in the example. However i'm not sure how that translates when using the models on TF hub. TF hub doesn't provide a link to the labels like is used in the example you cited.

Yes it does for example:

https://tfhub.dev/google/imagenet/inception_v1/classification/4
https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt

Link to post
Share on other sites
1 minute ago, zhnu said:

Okay, thank you. I'm really just a disorganized mess. I'm used to the flow of most npm packages where the node usage is described in the package page. Whereas with this the information is scattered and there is no copy/paste example code for each model.

 

So I tried running the example code above for the inception model, and I got:

TypeError: Cannot read property 'dataSync' of undefined

at this point in the example: 

 const extractClassesAndMaxScores = function (predictionScores) {
  console.log('calculating classes & max scores')

  const scores = predictionScores.dataSync()
...

 

Computers r fun

Link to post
Share on other sites
6 minutes ago, TheNuzziNuzz said:

Okay, thank you. I'm really just a disorganized mess. I'm used to the flow of most npm packages where the node usage is described in the package page. Whereas with this the information is scattered and there is no copy/paste example code for each model.

 

So I tried running the example code above for the inception model, and I got:


TypeError: Cannot read property 'dataSync' of undefined

at this point in the example: 


 const extractClassesAndMaxScores = function (predictionScores) {
  console.log('calculating classes & max scores')

  const scores = predictionScores.dataSync()
...

 

predictionScores is going for some reason undefined to the function
Link to post
Share on other sites
9 minutes ago, zhnu said:

predictionScores is going for some reason undefined to the function
        const prediction = await model.execute(tf.node.decodeImage(frameData).expandDims(0).toFloat())
        const index = prediction.as1D().argMax().dataSync()[0]
        console.log(labels[index])

Okay I got this to work, I turned https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt into an array. 


sadly the results are completely inaccurate and nonsense. Thank you for your help!

Computers r fun

Link to post
Share on other sites
1 minute ago, TheNuzziNuzz said:

        const prediction = await model.execute(tf.node.decodeImage(frameData).expandDims(0).toFloat())
        const index = prediction.as1D().argMax().dataSync()[0]
        console.log(labels[index])

Okay I got this to work, I turned https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt into an array. 


sadly the results are completely inaccurate and nonsense. Thank you for your help!

The results have a score you have to look at the score and decide the threshold, but yeah is very easy to have false positives.

Link to post
Share on other sites
3 minutes ago, zhnu said:

The results have a score you have to look at the score and decide the threshold, but yeah is very easy to have false positives.

Is there any such way to get locally run models to be as accurate as google's Cloud Vision?

Computers r fun

Link to post
Share on other sites
Just now, TheNuzziNuzz said:

Is there any such way to get locally run models to be as accurate as google's Cloud Vision?

You can try different models see what works best, but I doubt you can beat google cloud vision.

Link to post
Share on other sites
23 minutes ago, zhnu said:

You can try different models see what works best, but I doubt you can beat google cloud vision.

I want to use deeplab, but it keeps giving me 

(node:3128) UnhandledPromiseRejectionWarning: Error: The dtype of dict['ImageTensor'] provided in model.execute(dict) must be int32, but 
was float32

with

        const canvas = createCanvas(720, 640)
        const ctx = canvas.getContext('2d')
        ctx.drawImage((await loadImage(frameData)), 0, 0)
        model.segment(canvas)

 

Computers r fun

Link to post
Share on other sites

I figured it out! Deeplab only works with version 1.3.1 of tensorflow. I downgraded my packages and it is working amazing. 

Computers r fun

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×