I work in this field, and I wrote my bachelor's thesis here, not with humanoids but with VLAs (think chatgpt connected to a robot arm)
It's certainly not there yet for anything practical, there's also certain bits and structures that don't have accurate names during construction, and it is important to keep that in mind - so a robot is unlikely to understand what it means to say "put the left bit of this box onto this right bit" due to ambiguity, a human would understand that
Plus we have no good reliable accuracy testing data in most cases (most tests occur on a few demos, but that isn't a good representation of how must things work), popular benchmarks, such as libero have been saturated, and nearly everything gets 95% there, most companies and researchers have their own benchmarks here.
Plus companies lie alot, and do very dangerous things in thier videos, I.e. these robots should not be standing very close to humans, because of being dangerous.
There are also legitimate concerns of misuse of these robots that need to be accounted for, misuse does not have to be warfare, but can be as simple as confusing it while it is cutting tomatoes with a knife.
Turning doorknob is easy, and fail recovery is also being worked on, but we don't have reliable statistics anywhere on that. The hard part is on practical things, as in when placing bricks or attaching a part during manufacturing it needs to ensure that it is aligning everything correctly....and that's hard, while it is impressive, it is very irresponsible to keep humanoids at home (people are irresponsible when untrained), for example, lawnmowers injure about 6400 people a year...and that is not an everything machine.
Humanoids in general are...not appealing in specific, due to maintainable of joints, complexity, but robot arms in particular, expecially on wheels (check mobile aloha), are likely to be able to do tasks such as clean up in hotels, after a guest had left, or replace some cooks in restaurants (if their work is consistent)
This is absolutely not true check out Sunday robotics and their robot that folds clothes. Sunday robotics says it does holding of clothes correctly 99% of the time that is real world usage today. They even have three hour video of the robot folding the clothes.
No, the above is absolutely exactly true. Robots folding clothes and tying shoelaces etc is nothing but a tech demo at this point.
It's difficult to grok this because if you watch a human folding a t-shirt, you can reliably predict that the same human will fold a different t-shirt just as well, and in fact be perfectly capable of folding a wide variety of other clothes items as well. Not so for robots. With robots, what you see is precisely what you get. If you see a robot folding a t-shirt, all that means is that that particular robot can fold that particular t-shirt. The state of the art today is that the same robot cannot be expected to be able to fold a different t-shirt.
For example, see this article about Mobile ALOHA at Google. There's a passage where the visiting, awe-struck, journalist asks whether the robot he's just seen tying up a pair of shoelaces can tie up his own shoe.
“If I gave it my shoe,” I ventured, “would it just totally fail?”
“We could try,” Tompson said. I removed my right sneaker, with apologies to
anyone forced to handle it. Tompson gamely placed it on the table, while
Driess reloaded the policy.
“To set expectations,” Driess said, “this is a task that is thought of as
being impossible.”
Tompson eyed his new experimental subject with some trepidation. “Very short
shoelaces,” he said.
The policy booted up, and the claws set to work. This time, they poked at
the shoelace without getting a grip. “Do you give consent for your shoe to
be destroyed?” Driess joked, as the hands grabbed at the tongue. Tompson let
them try for a few more seconds before hitting the Failure pedal.
As to ACT-2 which basically uses the same techniques as ALOHA (imitation learning) far as I can tell, that's a commercial product and the information they give on their site is difficult to parse. E.g. they say they have 99.1% ±0.3 success rate, 778 successful folds and 9 garment types which is low enough to engender some trust they're not trying to inflate their numbers, but they don't say whether they trained on the garments used in evaluation or not. Chances are they did, because that's the current limit of the technology, i.e. if the garment being folded is unseen (as opposed to the environment, which they tout) then performance is basically random. So either ACT-2 have a major breakthrough that is a few leaps and bounds away from the current state of the art, or you've just watched a tech demo.
The fact that they only advertise "9 garment types" though is a big hint: they have the same problem with generalisation as everybody else at this point in time.
You make some very good points about the ACT-2 if it’s nine garment types that’s still very acceptable but the point you bought up about them training on exactly those pieces of garments is a possibility.
Cheers. You can see that's what they did if you look at the 4th video on the page, under the figure titled "Quality Remains High Across Garment Types" right below the paragraph that starts with "To put these scores in context". Sorry, I have no idea how to link to that video specifically.
In the left half of the video you can see that the robot is (trying to) exactly match the folds of the human in the right half and it's doing so while folding the exact same garments on the exact same surface.
The right half of the video is not a training demonstration, I don't think, since the robot is trained by teleoperation AFAICT (it needs to because it must use its head-mounted camera to control its movements) but that just underlines the degree to which their training regime is exactly copying the movements of a trainer, on the same garment, in the same environment.
This is a limitation of the training approach, by RL. With RL you learn a mapping between sets of pixels (as in the video that comes in through the robot's camera) and robot actions (as in actuator commands). What that means is that once a policy is trained and the robot is deployed, if the input pixels are significantly different than the input pixels at training the robot doesn't have a policy that matches the input pixels and so it can't find the right actions to take. So they have to keep the training and deployment garments and even the environments the same, or as similar as possible.
You can see some more evidence of this in the video right under the paragraph with the title "Hill-Climbing Reliability Through Post-Training". The robot at the front of the video, with the bright red trim, is shown trying to fold a grey t-shirt with white flower decorations and a frilly hem (how adorable <3). But you can tell it fails because the video stops before the robot has completed the fold. If it could complete it, you can rest assured that the video would be showing off the entire folding sequence as it does for the robot with the green cap on the other side of the bed.
That paragraph is making a claim about a "post-training" regime that's supposed to improve generalisation but it leaves more details to a "separate technical post". So I can't tell what it's supposed to be doing, but I don't think it's working.
When I watch videos like that I always remind myself that a) I'm watching a tech demo created to attract investment and b) I've watched way too many of those, going all the way back to the Boston Dynamic videos of Robot Dog or of Atlas doing backflips and yet the state of the art hasn't really budged since. Such videos make it easy to overestimate the state of the art in autonomous robotics and in fact are meant do precisely that: play up robots' true capabilities. It's just impossible to say anything about a robot's general capabilities by watching a few minutes or even a few hours of video. OtoH if you know what to look for you can tell everyone is basically stuck at the same level and trying the same things to escape it. The truth is robotic autonomy is several major breakthroughs away and nobody has any idea how to get there. So we'll be seeing many more of those tech demo videos in the years to come.
It's certainly not there yet for anything practical, there's also certain bits and structures that don't have accurate names during construction, and it is important to keep that in mind - so a robot is unlikely to understand what it means to say "put the left bit of this box onto this right bit" due to ambiguity, a human would understand that
Plus we have no good reliable accuracy testing data in most cases (most tests occur on a few demos, but that isn't a good representation of how must things work), popular benchmarks, such as libero have been saturated, and nearly everything gets 95% there, most companies and researchers have their own benchmarks here.
Plus companies lie alot, and do very dangerous things in thier videos, I.e. these robots should not be standing very close to humans, because of being dangerous.
There are also legitimate concerns of misuse of these robots that need to be accounted for, misuse does not have to be warfare, but can be as simple as confusing it while it is cutting tomatoes with a knife.
Turning doorknob is easy, and fail recovery is also being worked on, but we don't have reliable statistics anywhere on that. The hard part is on practical things, as in when placing bricks or attaching a part during manufacturing it needs to ensure that it is aligning everything correctly....and that's hard, while it is impressive, it is very irresponsible to keep humanoids at home (people are irresponsible when untrained), for example, lawnmowers injure about 6400 people a year...and that is not an everything machine.
Humanoids in general are...not appealing in specific, due to maintainable of joints, complexity, but robot arms in particular, expecially on wheels (check mobile aloha), are likely to be able to do tasks such as clean up in hotels, after a guest had left, or replace some cooks in restaurants (if their work is consistent)