Hacker Timesnew | past | comments | ask | show | jobs | submitlogin

I wanted to test Gemini's code generation so I asked it for a bash script iterating through an array of directory names and executing a command for each one.

It got it wrong. The command was generated outside of the for, and never updated inside the loop effectively making it useless.

Luckily I know bash so I spotted it immediately. But I wonder how it's "removing programming language barriers" when one that does not know its output language can not spot even such a glaring issue.



That’s a bash problem, not LLM.

I’ve done similar tools with Go and haven’t had any issues.

One goes through a tree of dirs, finds zip files with images, unzips, benchmarks which image format is the most compact and creates a new zip with that.

Got it right the first go (hehe pun)


I think this is more of a criticism of bash than of Gemini.

Also, I out-of-hand reject any criticism of an AI that specifies only the brand ("ChatGPT") and not the specific model version, settings, etc...

So, for example, Gemini 2.5 Pro (Jan 2025) with temperature 0.15 prompted with: "Write a short, idiomatic bash script that takes a file containing a list of directory names, changes to each one, runs a command, and then returns to the original working directory."

Spits out this:

    #!/bin/bash

    # A script to run a command in a list of directories.

    # Exit if the wrong number of arguments are provided.
    if [[ $# -lt 2 ]]; then
      echo "Usage: $0 <file_with_dirs> <command> [args...]" >&2
      exit 1
    fi

    DIR_FILE="$1"
    shift # The rest of the arguments ("$@") are now the command.

    # Ensure the directory list file exists and is readable.
    if [[ ! -r "$DIR_FILE" ]]; then
      echo "Error: Cannot read file '$DIR_FILE'" >&2
      exit 1
    fi

    # Read the file line by line.
    while IFS= read -r dir; do
      # Skip empty lines or non-existent directories.
      if [[ -n "$dir" && -d "$dir" ]]; then
        echo "==> Entering '$dir'..."
        # Run command in a subshell to keep the 'cd' local.
        # '&&' ensures the command only runs if 'cd' succeeds.
        (cd "$dir" && "$@")
        echo "<== Finished with '$dir'."
      else
        echo "--> Skipping invalid directory: '$dir'" >&2
      fi
    done < "$DIR_FILE"

    echo "All done."
That worked for me, but I was testing it in WSL 2 where I got a gibberish error... which was because I edited the file in Windows Notepad and the line endings were confusing bash. Gemini helpfully told me how to fix that too!

Something that I found amusing, and again, is a criticism of bash instead of the AI, is that this fails to process the last line if it isn't terminated with a \n character.

PS: This is almost a one-liner in PowerShell, and works with or without the final terminator character:

    Push-Location
    Get-Content dirs.txt | cd -PassThru | Foreach-Object { echo "Hello from: $pwd" }
    Pop-Location
Gemini also helped me code-golf this down to:

    pushd;gc dirs.txt|%{cd $_;"Hello from: $pwd"};popd


> I think this is more of a criticism of bash than of Gemini.

I can write correct bash; Gemini in this instance could not.

> Also, I out-of-hand reject any criticism of an AI that specifies only the brand ("ChatGPT") and not the specific model version

Honestly I don't care, I opened the browser and typed my query just like anyone would.

> PS: This is almost a one-liner in PowerShell, and

Wonder how this is related to "I asked Gemini to generate a script and it was severely bugged"


> typed my query just like anyone would.

Yes, well... are you "anyone", or an IT professional? Are you using the computer like my mother, or like someone that knows how LLMs work?

This is a very substantial difference. There's just no way "anyone" is going to get useful code out of LLMs as they are now, in most circumstances.

However, I've seen IT professionals (not necessarily developers!) get a lot of utility out of them, but only after switching to specific models in "API playgrounds" or some similarly controlled environment.


> Yes, well... are you "anyone", or an IT professional? Are you using the computer like my mother, or like someone that knows how LLMs work?

I have more than 15 years of programming experience. I do not trust the output of LLMs a single bit. This just proved my point. I honestly don't care if I used the "wrong" model or the "wrong" query, which was already quite descriptive of what I wanted anyway.

No need to get super defensive, you can keep spending your time playing code golf with Gemini if you want. My experience just corroborates what I already thought; code generation is imprecise and error prone.


> I honestly don't care if I used the "wrong" model or the "wrong" query

If you used the wrong SQL query, would you expect the right answer?

If you used the wrong database, would you expect your app to work well?


Not even remotely comparable


  for dir in $(cat dirs.txt); do ( cd "$dir"; echo "Hello from $(pwd)" ); done


Unbelievable how long and convoluted the other answer is, and that it is presented as proof that the AI provided a good solution.


I asked for a "script". Asking for a one-liner does just that, with no input validation, comments, etc...

Fundamentally, bash is just... verbose.

I.e.: Here's the same task implemented in two scripting languages:

PowerShell is 5 lines of code: https://learn.microsoft.com/en-us/azure/virtual-machines/ena...

Bash is several pages: https://learn.microsoft.com/en-us/azure/virtual-machines/ena...


That's more of a reflection of the environment than the scripting language. On a mostly bare linux (debian netinstall or alpine), you are left with loads of text to parse. But as soon as that script becomes unwieldy, then the next option is write an actual program. Windows can afford to do that, because there's only few version out there. But there are lots of different Linux installations. And even the kernel is not guaranteed to be vanilla. So you're either write a script like this, or you go find programs that can help you out.


Out of curiosity isn't () spawning a subshell?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: