That goes against the definition of the word complication. To complicate something is to combine and intertwine it with other concerns. To fold them together is to complicate them.
To generify a function is to complicate it with the ability to accept multiple types rather than just one.
There are totally great use cases for generics but all the cases I’ve seen are in library code not in general application programming which is where most developers spend most of our time. But of course the shiny toy can be hard to resist and so generics are often overused and abused.
As a general rule, if you're referencing the dictionary definition of a word to make your point, you're just playing semantic games.
You know what people also find complicated? Hundreds of lines code being repeated with superficial edits because of golang's lack of ability to abstract higher-level ideas. It's a stupid toy example, but for a very large number of people
nums.take(20).select(&:odd).reduce(&:+)
is less complicated than
sum := 0
for i, v := range nums {
if i > 20 {
break;
}
if v % 2 == 1 {
continue;
}
sum += v;
}
The former is less complicated for those people (including me) because it expresses high-level intent rather than low-level implementation. Multiplied over a large code-base, the ability to express intent adds up to an enormous saving in mental overhead rather quickly. You can always drill down and focus on implementation where it matters, but with golang there's almost zero ability to separate a program's detailed implementation from a high-level description of what you're actually trying to accomplish.
The Ruby version of this is also less bug-prone (did you spot the bug in the go example?). And it's also easier to apply to new contexts: the former works automatically for infinite enumerations.
There are two bugs in the code, and your (intentional?) use of non-idiomatic Go is the cause of one of them. Since you don't mention it in your follow-up comment I assume you didn't mean to write it in the first place.
Still, there is a middle ground that remains both efficient and readable:
sum := 0
for _, v := range take(20, nums) {
if v % 2 == 1 { // or if odd(v), if you like
sum += v
}
}
I'd venture most of the clarity come from the generic take (specifically, being able to implicitly take min(len(nums), 20)), not the generic filter/reduce. The second point of clarity comes from Ruby's dynamic method dispatch and large method set on numeric types; no thanks. Only as a third-tier aspect does select/reduce come into play, and I think that one is much more questionable (e.g. reduce or fold, and either way how do I explain this name to new programmers? -- and &:+, what a messy identifier).
I legitimately cannot spot the bug. Can you write out the equivalent Ruby code in Go pre-generics? I really did try but I honestly cannot understand what your Ruby code is supposed to do. I don't know what select and take are supposed to do, and I can guess at reduce but the symbols in the method are utterly arcane to me. Does Take grab exactly 20 items or does it do something like nums[:21]? Depending on how Take is implemented it could be an off-by-one error in your loop? Usually in this case I'd check for equality instead; that's the only thing that looks off to me in your Go code.
In my opinion the Go code is dead simple and unambiguous with no nuance hidden away in methods whose exact semantics I certainly don't have memorized. I'd have to look at 3 different function signatures to figure out what those Ruby methods do every single time I was reviewing code like this.
The code is equivalent, except the golang version accidentally loops an extra time.
`&:meth` is just Ruby syntax for "a function that calls the `meth` method on its argument". So `&:foo` is shorthand for `func(x) { x.foo }`.
`take(n)` just returns the first `n` elements.
`select(fn)` iterates for every element for which the function passed to it returns true.
`reduce(fn)` combines the previous iteration with the result of calling the provided function on the next iteration.
Thus `take(20)` gets the first 20 elements, `select(&:odd?)` iterates over every odd value in what's left, and `reduce(&:+)` adds every remaining element together.
> In my opinion the Go code is dead simple and unambiguous with no nuance hidden away in methods whose exact semantics I certainly don't have memorized.
These functions and Ruby's `&` syntax took a few sentences to explain. They are fundamentally not that complicated, although nobody would expect you to understand what they did before you've seen them just like any other function call.
You didn't know what they do, which is fine. But now you do, and it is completely normal for people to assume that you should be capable of working with them in the same way that someone would expect you to write 2 * 5 instead of writing 2 + 2 + 2 + 2 + 2. Writing the full `for` loop in my example is the equivalent of the latter, with the equivalent pitfall that it's really easy to accidentally write an extra sixth addition when you only meant to put five of them.
> In my opinion the Go code is dead simple and unambiguous with no nuance hidden away in methods whose exact semantics I certainly don't have memorized. I'd have to look at 3 different function signatures to figure out what those Ruby methods do every single time I was reviewing code like this.
No, you wouldn't, for the exact same reason you don't look up the documentation for `for` or `range` or `if` or `*` every time you use them. They are simple, straightforward abstractions that any programmer will have more or less fully internalized given fifteen minutes of playing around with them, and which you will likely use multiple times a day in any language that supports them.
I can assure you that bordering on 0% of programmers who regularly use languages with these functions did not understand the example.
There was a time when I, too, did not know what to make of functions like these. I saw someone write a similar comment and I thought "that's totally inscrutable". And then I learned what those functions do and I now use them almost literally every single day. They are amongst the most useful and universal abstractions I have ever come across.
I think you overestimate my ability. I've spent > 10 years programming in languages that have features like this (take and select excluded I think? Or maybe they had different names?) and I've just never internalized functions that are generic like this. Usually what I end up doing is opening up a REPL or making a toy program so I can iteratively see what happens to the collection. This probably makes me a bad programmer but I'm just being honest: It's too much cognitive overhead for me when I'm trying to spend the rest of my brain power on keeping track of the actual problem I want to solve -- which usually means interpreting code written by 30 other engineers over long period of time (read: it's a mess).
I was reflecting on this and the one language where I don't feel this way is SQL. I'd consider it my strongest language, and it's an extremely functional language. I have no trouble deciphering complex SQL, but it takes me an enormous amount of time to figure out what happens in those long function chains in general purpose programming languages. I'm not sure what it is about SQL that makes me feel this way, but maybe it gives some hint why our brains seem to work in different ways?
I think you underestimate your ability. As a programmer, you've already learned and internalized abstractions that are far more complicated than anything here.
`select` is often aliased to `find_all` and that's what it does: finds all elements matching what's passed to it. `take` is sometimes aliased to `first`. Expanded ever so slightly:
nums.first(20).find_all { |n| n.odd? }.reduce { |sum, n| sum + n }
This reads: take the first twenty elements, find all the ones that are odd, and reduce by adding them together. The only "clunky" bit here is the word "reduce" which IMO there isn't an equivalent common English word for that gives a good intuition for what it does.
It might not look like it if you haven't internalized these abstractions, but they reduce cognitive load. Dramatically. You don't have to read through a complicated set of conditionals and control flow statements in an explicit loop, you can read the bits entirely linearly and (almost) in straight English. Once comfortable with these, you can quickly glance at pretty much any expression like this and know immediately both what it does and have extreme confidence it doesn't contain bugs, because there just isn't anywhere for bugs to be.
Don't sell yourself short. Take the time to learn these, and you will be a better programmer for it.
Perhaps it comes down to a difference in where we spend most of our time? I'll give you an example.
I once spent a full day debugging a problem that came down to the implementation details of .zip. The author had assumed that .zip would add extra null elements to the output array if the inputs didn't match in length, which is sadly not the behavior of our programming language. We determined this was the bug after breaking out the REPL and running line by line because it was hard for us to visualize exactly what was happening in functional methods like these (there were more around the .zip call). We ripped out the .zip and turned it into an explicit for loop because we wanted the behavior for the case of "these arrays are different length" to be extremely obvious to the reader. The author and myself probably learned that zip had this behavior at some point, but its terseness hid a ton of nuance in code review that we decided we cared about later on in a way that explicit looping did not.
So, I get that internalizing functions like this can reduce cognitive load in some cases and it's certainly shorter. However, I spend a large percentage of my time looking at code where small semantics like the above matter a great deal. What happens if the length of this array is less than 20? What is the default return value if there are no items: None or 0? When I loop over something, it's often extremely important -- something that operates on an absolute ton of elements, altering its behavior is a big change in business logic type of stuff. Too often we've run into edge cases like the above that just flat out need to be explicit.
I think if I were working on smaller teams/codebases with more homogeneous experience levels I might feel differently. On my current team I will take the tradeoff of "the average function takes a bit longer to parse" if it means that everyone can reason about any given bit of code without trouble. We're trying to minimize how bad things can be, e.g. never have multiple programmers sitting around a computer trying to figure out what the bug is in a nested list comprehension with gratuitous use of function chaining. We use Go over other languages nowadays because we believe that for our team, explicitness results in the lowest cognitive burden on a global level. I think it's fine that other languages make other choices -- sometimes I program in Haskell for fun -- but if I come back to something I wrote long ago, I always break out the manual to remind myself what exactly certain expressions do.
> We ripped out the .zip and turned it into an explicit for loop because we wanted the behavior for the case of "these arrays are different length" to be extremely obvious to the reader.
This could have been accomplished by just extending the shorter array to the length of the longer one with no loss of clarity (and likely greater clarity, as now I don't have to read your custom implementation of `zip` every time I read this call site).
The broader point is that you found a bug where someone used a function incorrectly and instead of fixing the usage of it, you simply wrote the function inline. This same story could have been with any function call, but for some reason it seems you think that iterator methods are special and different somehow? Any function can be called incorrectly, but the solution isn't to just universally replace function calls with inline equivalents. You had a bad experience with not understanding one of these types of functions, so instead of taking a moment to internalize what they do, you decided to swear off of them entirely? I honestly, genuinely cannot understand this perspective.
> What happens if the length of this array is less than 20?
In 100% of implementations I've ever encountered, it returns fewer than 20 elements. If you want exactly 20, call `take` and then pad its length with whatever-valued elements you need. Explicit.
> What is the default return value if there are no items: None or 0?
Up to you! Pass the default return value as the first argument to the `reduce` method. Explicit.
These aren't deep and particularly confusing semantics around these methods. These are just garden-variety "I instinctively avoid these functions so I don't know the basics of how they work" types of questions. Making the answers to these questions explicit does not require splatting out the entire contents of their function definitions inline. That's not explicit, it's verbose.
> the average function takes a bit longer to parse
Code is read dozens if not hundreds of times more often than it's written. Code must be written to minimize the effort needed to understand it. The entire point of functions is to assist with this. The entire point of these specific iterator functions is that they do a phenomenal job of this, to the point where virtually every single programmer who works in languages with these idioms will understand what you mean when you say you're mapping an array.
You're absolutely capable of the same, but for some reason you've decided that these functions are magic and scary and should be avoided. They're not, and regularly avoiding them actively decreases the clarity of your code and is far more likely to increase your bug count than decrease it.
> if it means that everyone can reason about any given bit of code without trouble.
Where is the floor on this? One engineer decides that `map` or `select` or `all` isn't worth bothering to learn, so nobody gets to use them? What if they decide a `for x := range y` is too much work, does everyone go back to `for x = 0; x < y.len(); x += 1`?
These functions are basic. They aren't fancy functional magic that only Haskell wizards will ever hope to comprehend. They are used in an enormous variety of languages where their users overwhelmingly find them to be a net increase in clarity while eliminating the possibility of entire classes of common derpy bugs, no differently than `for x := range y`.
Have you thought about the amount of allocations and copying your chained generic map-reduce thing would cause? Go for loops are good because they are simple, even if you have to use more than one line.
Generally direct refutation of a central point is a constructive argument. Here’s another example of direct refutation:
You’ve given a strawman argument, specifically you’ve given one implementation which has abstracted the details (we don’t see the code for take, select and reduce). That’s just an arbitrary decision you’ve made, the equiv go example you could have posted might be:
You’ve presented these different levels of abstraction and then argued against a point that wasn’t made. A strawman argument.
In the interest of steelman-ing your argument, the interesting difference would be in the comparison of implementations of take() or select() or reduce() - but ruby is a dynamic language so there’s not really a comparison to be made.
We can still say how we might approach Take() or Select() or Reduce() or Sum() though if we need them to be generic over argument types - in the absolute worst case (so not using go generate to help us here or an interface or the new generics functionality) in the worst case e we would have repeated definitions of these functions. Code that any junior developer will be able to safely reason about and change. Code that has utterly obvious risks (you might introduce a differing behaviour in one implementation of Take() for example) - so obvious that it’s trivial to defend against with nothing more than generative testing. Again, painfully simple code. Zero cleverness. Any developer of any experience level can quickly make a valid change.
> Generally direct refutation of a central point is a constructive argument.
Selecting your own definition for a word and basing an argument around the definition you chose for it is not direct refutation of a central point. Again you appear to just want to play games where you get to declare yourself the Internet Argument Winner and pat yourself on the back instead of actually giving a shit about the perspectives of those who disagree with you.
> You’ve presented these different levels of abstraction and then argued against a point that wasn’t made. A strawman argument.
Those functions don't exist in go, and up until generics were just added, they couldn't be without copy/pasting their implementation for every single array type you wanted to implement them for. You're essentially making my point for me in that the only way these functions can be written now without resorting to copy/paste in every project that wants to use them is thanks to generics.
I presented this argument because it is a common refrain in the Go community that functional iterators like map, filter, reduce, and take are unnecessary and add complexity and are unnecessary. Far from a straw man, this is a direct example of a case where people have pleaded for generics while people like yourself have argued that it increases complexity. You can find examples of those types of perspectives right here in this comment section.
And this is just one example of an area where golang has historically foisted complexity onto its users rather than solve it internally.
I think the code you linked shows exactly why generics are needed for these kinds of methods. By introducing a single generic, you could reduce it from 500 lines to 20, and make it work for all types, not just the built-in ones. It would also remove the need to have different method names.
I think that makes it less complex, as the developer doesn't need to think about the exact underlying type of the array when calling Take (which is irrelevant to its implementation).
> in the worst case we would have repeated definitions of these functions. Code that any junior developer will be able to safely reason about and change
It's also code that many junior developers will forget to change in all the places when they fix a bug in one of them.
>> Code that has utterly obvious risks (you might introduce a differing behaviour in one implementation of Take() for example) - so obvious that it’s trivial to defend against with nothing more than generative testing.
Really obvious risks are usually easier to handle than more obscure ones.
Whether or not you like the Ruby version is beside my point, which is that which one of those is "more complicated" is a matter of perspective. The Ruby one is almost strictly a wrapper around the golang one so it does add absolute complexity. But the golang one is relatively more complex, because the Ruby version uses higher-level equivalents, and those abstractions are good enough that I don't ever have to actually reason about what happens underneath the covers.
Put another way, even the golang version is an abstraction around an absolutely massive amount of hardware and electrical engineering complexity that you virtually never have to think about. The absolute complexity of what happens when you compile and execute those instructions is extreme to the point that no single human or even room of humans collectively fully knows what's going on. And yet we manage just fine.
Yep, the `userNames` thing was my own dumb mistake while editing in a comment box and was not intended as a reflection of the language. The capacity/length issue is the actual bug I was intending to include.
But... also it's a mistake that's not even possible in the Ruby example due to not having to handle the "irrelevant" internal details of appending to the array, so maybe there's a point to it after all.
I grant you that the API for capacity and length when creating a slice in Go is bad because of this exact mistake, and I believe a few of the Go authors said they regretted it and would like to change it. However, it's odd to classify this as an advantage of Ruby, because pre-allocating memory for a large slice like this has been one of the biggest single performance advantages when we moved from Python to Go, and it's effectively a one line change. For known-to-be-tiny slices we don't even bother.
and its an optimization the ruby version doesn't necessarily have. you could have just as easily written:
var names []string
for i, user := range users {
if user.isAdmin {
names = append(names, user.Name)
}
}
and hilariously it likely still would be faster than ruby. I've managed large golang and ruby projects and the difference in maintenance work between the two is insane and its not a good look for ruby.
That goes against the definition of the word complication. To complicate something is to combine and intertwine it with other concerns. To fold them together is to complicate them.
To generify a function is to complicate it with the ability to accept multiple types rather than just one.
There are totally great use cases for generics but all the cases I’ve seen are in library code not in general application programming which is where most developers spend most of our time. But of course the shiny toy can be hard to resist and so generics are often overused and abused.