Frigate+ Base Model 2024.1 Update #11106
Replies: 30 comments 68 replies
-
Very excited to try the new model, there has been one plant in particular out the front of my home that supposedly looks exceptionally human 😅 |
Beta Was this translation helpful? Give feedback.
-
Hello , Sorry for the questions, thank you very much for all your work! I can't wait to try it out! |
Beta Was this translation helpful? Give feedback.
-
Very excited to try this, really appreciate the hard work you put into this. I'll provide feedback over the next week :) |
Beta Was this translation helpful? Give feedback.
-
I’ve been eagerly awaiting this, 2024.0 was great and set a high bar. Just got my 2024.1 completed and configured in and super keen to see how it fares. edit: I’d love to see a write up / summary of what changes you made to training sample selection this time. |
Beta Was this translation helpful? Give feedback.
-
I'm wondering how we could get false positive statistics for our own data - like you summarised in your intro. |
Beta Was this translation helpful? Give feedback.
-
Honestly I was very underwhelmed by my first model using 2024.0, even after tuning it wasn't a lot better. |
Beta Was this translation helpful? Give feedback.
-
Tried my first + model, its not working. I get this in the logs: 2024-04-26 03:19:54.587111853 Process detector:ov: |
Beta Was this translation helpful? Give feedback.
-
I have statues of lions on my gate pillars, which are often mistaken for either a person or a dog. (Interestingly, they are almost never mistaken for a cat.) From the camera's perspective, people almost always walk behind the statues. You mentioned that to reduce false positives, we should submit true positives in that area along with the false positives. Now, if a person walks behind one of these statues and I submit it as a true positive with the statue overlapping the person, would that confuse the model? |
Beta Was this translation helpful? Give feedback.
-
So in this circumstance where the Frigate+ identified box is too big should we mark this not a package and draw a tighter bounding box? |
Beta Was this translation helpful? Give feedback.
-
So my initial results are that false positives are indeed way down, but false negatives are up as well (which is obviously a bit of a bigger deal). This might be because I hiked up my min score on the previous model, so I'm gonna try to reduce that and see how it goes. The camera with the most notable negative impact in my sole newer (to me) fisheye camera.... its detecting near nothing now. I don't have a whole lot of training on it either, so I'll have to see if I can drop the minimums pretty low and start training it. I do have it using the raw stream, with no de-warping, so I don't know if that is a proper goal or if there should be a different model for this unusual camera format? I just didn't want to create multiple additional streams as I have so many cameras already, and I'm near the limit of what my hardware can support without more RAM. |
Beta Was this translation helpful? Give feedback.
-
Looking pretty good so far with the new model. Still some false positives and misidentifications here and there, but the false positives have been noticeably reduced, making it much easier to scroll back through events. The only identification that doesn't seem to have been reduced is deer, but most of those are relatively low confidence, I'll just set the minimum higher. The stationary object tracking seems to be working much better for me with parked cars. It was losing the car-ness of parked cars many times a day previously, now it's pretty solid. I frequently have two cars that overlap from the camera's perspective, I could watch the detection going nuts hopping back and forth between which one of them was a car and which wasn't. I did notice one particular type of new false positive showing up. One of my cameras can see the street through some (sparse) tree branches. I've been submitting plenty of true positives of cars driving by seen through the branches, Frigate has been doing a fine job of catching those. Now it seems I've trained this new model that cars, and branches blowing in the wind, are cars. I'm submitting those false positives now, hopefully it'll go back to normal with the next model I train. I think my biggest remaining problem model-wise is that squirrels and foxes are always running by firing off a confident stream of "cat - dog - cat - dog - cat - dog" events. |
Beta Was this translation helpful? Give feedback.
-
Thanks for your excellent work!! One question, I've submitted some images with dog labels from different cameras, day/night etc but my frigate is not detecting any since I started using this custom models. Should I keep uploading and labeling dog images? I've lowered the score for dogs to 40-50% and even like this no detections. Thanks. |
Beta Was this translation helpful? Give feedback.
-
At risk of resurrecting a horse to beat; was a list of labels for future models firmed up? I'm keeping my raised vegetable gardens squirrel free with homeassistant, a smart plug and powered water valve to spray them if frigate detects them in frame. Currently the squirrels are detected as birds, so I am hesitant to update the model without a squirrel tag or something to keep this working as it is currently. Not a complaint in any way, this project is great and all your hard work is very appreciated. 🥇 |
Beta Was this translation helpful? Give feedback.
-
I'm seeing (subjectively) a huge drop in false postives particularly at night. I'm also seeing a small number of new false positives (new in new places). So I guess this is 10 steps forward and 1 back. Brilliant. |
Beta Was this translation helpful? Give feedback.
-
Regarding face detection, I've added face as object:track and added
shouldn't that do the trick? |
Beta Was this translation helpful? Give feedback.
-
~48 hours on 2024.1 and this is the only false positive I’ve seen yet… a friendly spider-person 😂 but a bit before that, we did get a nice unexpected true positive (100% confidence?? 🧐) of the Jeep down there, despite the obscurities. So far so good!! I have 6 cameras with roughly 2500 images submitted. |
Beta Was this translation helpful? Give feedback.
-
Hello, I just like to add a feedback for this new model, it's way better than the one last month in false positive and it detects really better also at night. I have 2 cameras at the front of the house, usually only one camera can catch me at a time, now basically the two cameras will catch me in the same time wherever my position is. I still have some false positives, but it's really way better than before ! I'm just having the feeling that if more frigate users upload and train the model, it'll improve more and more everyday. Well done ! |
Beta Was this translation helpful? Give feedback.
-
My subjective experience with the new 2024.1 is very good and the improvement over 2024.0 is substantial. I was able to lower my thresholds, and still getting noticeably lower false positive rate. |
Beta Was this translation helpful? Give feedback.
-
Hi, I just got my model based on newest version. 10 cameras were trained with average of 15 images each and I am getting a lot of false positives. As an example, for below (camera was trained in 30 images) the model wrongly detected person, and did not detect a car. I will be now gathering and sending all those false positives and generating new model in next days. |
Beta Was this translation helpful? Give feedback.
-
agreed with all others here, substantial improvement with 2024.1 over 2024.0. this one is almost perfect |
Beta Was this translation helpful? Give feedback.
-
Are models applied to all cameras or individual ones? My doorbell has worked perfectly for the past year but a new camera on the side of my house is throwing up false positives nearly daily - 76% chance of a plant pot or garden chair being a person sort of thing where I expect it to be mostly silent on the notifications. I have been uploading a load of positive/false positive verified images in hope that a model might solve it and I can use it more like an alarm notification. I don't want to cause any problems for the perfectly fine doorbell camera though! |
Beta Was this translation helpful? Give feedback.
-
one question that I dont see addressed in the docs--do we need to label objects in submitted images that we dont intend to track in our own setups? For example, its very cumbersome to label licenses plates in many of my cameras but this isn't an object I'm interested in tracking and I dont have it configured. Should I be labeling it in Frigate+? |
Beta Was this translation helpful? Give feedback.
-
I have a pretty human looking lamp. I can see why this has been so difficult to convince the model is not a person. I sent about 300 images so far and only had mild success. This new base model has made worlds of difference. So far after about a week I have had zero false positives! |
Beta Was this translation helpful? Give feedback.
-
Question on Send to Frigate+. If we use this option, and then go over the Positive matches, should we correct the bounding boxes to be more fitting with actual objects? making them smaller to fit better. I guess that in case where its missing and not catching entire object we should absolutely correct it. |
Beta Was this translation helpful? Give feedback.
-
I haven't had a single false positive with 2024.1. With 2024.0 I got heaps. I ended up churning through model credits trying to train out the FPs, with mixed success. Will you make additional model credits available once we hit 0 available? I went through 7 or 8 semi-futile model credits with 2024.0. |
Beta Was this translation helpful? Give feedback.
-
2024.1 is working for me far better than 2024.0 ever did. Much happier with it. Still getting the odd false positive (will retrain on a new model soon) but noway near what it was. |
Beta Was this translation helpful? Give feedback.
-
I've just trained my first model (about 600 verified pictures) and so far so good, it's detecting all the things it was previously with higher scores and hopefully will continue with less false positives - nighttime is normally my cameras weakness. I did have a false positive earlier (rare during daytime on the standard model) and I can't figure out why it would remotely think it was a person (72% in the UI, 67% in the snapshot) but after reading some comments above it sounds like I might at least benefit from setting a
|
Beta Was this translation helpful? Give feedback.
-
I saw an improvement with the new model but I got some false positives that I already labeled on frigate+ a few months ago. Is that normal? Will I need to label things again when base model changes? |
Beta Was this translation helpful? Give feedback.
-
Hello, is the release date of the version 2024.2 known? |
Beta Was this translation helpful? Give feedback.
-
Anyone noticed a reduction in inference speed? I just moved away from the OV detector to the Coral detector. The inference with the original model with the Coral detector is 9ms, but with the Plus 2024.1, it's around 140ms. Is it expected? I remember reading somewhere that the Plus models would be more lightweight. |
Beta Was this translation helpful? Give feedback.
-
NOTE: The base model was updated 24 hours ago. If you recently requested a new model, check the base model version before requesting a new one.
The base model in Frigate+ has now been updated to 2024.1. All model requests going forward will use this version. In order to get a new model, you will need to submit a model request.
In addition to incorporating new images from Frigate+ users, this base model update was focused on reducing known false positives. To date, nearly 250,000 false positive examples have been submitted by users. The previous base model reproduces 26% of these false positives and the new base model reduces that to 11%.
I also updated the fine tuning process to increase the effect that your verified images have on your model. My fine tuned model now reproduces only 1.5% of my reported false positives.
Here are some stats showing the false positive reduction for my model.
According to every objective measure I have looked at, this should be a significant improvement in false positive rates, but I expect results to vary by user. I would appreciate some user feedback (both good and bad) after upgrading.
Beta Was this translation helpful? Give feedback.
All reactions