Hi,

Is there any way to Filter an Array ?

I have an array of 1000 elements.

Want to give an option to search a string in the array. Then it should
show only elements which contains the search string. 

Like SET FILTER To ("KIRON" $ CUST->Name)

Regards
Kiron

You have to make a copy of the array. Code below not tested!

aNewArray := FilterArray(aArray1000, {|x| "KIRON" $ x} )

FUNCTION FilterArray(aOriginal, cbFilter)
LOCAL aReturn := {}, I

  For I := 1 To Len(aOriginal)
    IF Eval(cbFilter, aOriginal[I])
      aAdd(aReturn, aOriginal[I])
    ENDIF
  NEXT
RETURN aReturn

On 8/6/2020 6:26 AM, Pascal Boivin wrote:
> You have to make a copy of the array. Code below not tested!
> 
> aNewArray := FilterArray(aArray1000, {|x| "KIRON" $ x} )
> 
> FUNCTION FilterArray(aOriginal, cbFilter)
> LOCAL aReturn := {}, I
> 
>    For I := 1 To Len(aOriginal)
>      IF Eval(cbFilter, aOriginal[I])
>        aAdd(aReturn, aOriginal[I])
>      ENDIF
>    NEXT
> RETURN aReturn
> 
Hi,

Just to confirm Pascal's concept works but out of curiosity I created 
code two filter functions with the first using a FOR/NEXT loop and the 
second using AEVAL expecting the AEVAL to be faster... NOT!

FUNCTION uArrayFilter1(aArray,bFilter)
LOCAL aFiltered := {},;
       nPnt

    FOR nPnt := 1 TO LEN(aArray)
       IF EVAL(bFilter,aArray[nPnt])
          AADD(aFiltered,aArray[nPnt])
       ENDIF
    NEXT nPnt

RETURN aFiltered

FUNCTION uArrayFilter2(aArray,bFilter)
LOCAL aFiltered := {}

    AEVAL(aArray,{|x|IF(EVAL(bFilter,x),AADD(aFiltered,x),NIL)})

RETURN aFiltered

For my test I created an "original" array with 1 million elements...

    FOR nPnt := 1 TO nSize
       aOriginal[nPnt] := STR(nPnt)
    NEXT nPnt

And then used a filter: bFilter  := {|x| "3" $ x }

The FOR/NEXT loop filter method took .48 seconds to process the array 
and the AEVAL method too almost twice as long at .84 seconds!  I Also 
tried a 3rd method which created a clone of the passed array and then 
used AREMOVE to remove elements that did not match the filter 
criteria... This was pathetically slow... I had time to get a cup of tea 
before it finished!!!

I'm still surprised that the AEVAL method was slower but perhaps somehow 
in the FOR / NEXT process the bFilter works more efficiently???

Thanks Pascal for the "food for thought"... Regards... Jonathan

jonathan.leeming@familycentre.org
Edmonton, Alberta, Canada

Maybe because with the FOR/NEXT everything is compiled except the
codeblock. With AEVAL it is evaluated at run time.

Jonathan,

If you deal with large arrays, with hundreds or thousands of items, you can 
speed this function up considerably by doing these two things:

1) Move all unnecessary calculations out of the loop, specifically the 
   "len(aArray)" function call. Replacing this with a variable that is 
   calculated before the loop is started will save you from having to 
   calculate the same value in every loop.

2) Create a large (enough) array up front -- and adjust its size if 
   necessary inside the loop -- and truncate it at the end. Dynamically 
   adding items to an array is comparatively very slow.

So, with those two changes, your code would look like this:

Function uArrayFilter3(aArray, bFilter)
LOCAL nItems    := len(aArray)
LOCAL aFiltered := Array(nItems)
LOCAL nFiltered := 0
LOCAL nItem     := 0
   for nItem := 1 to nItems
      if Eval(bFilter, aArray[nItem])
         aFiltered[++nFiltered] := aArray[nItem]
      endif
   next nItem
   ASize(aFiltered, nFiltered)
return (aFiltered)

If memory usage rather than top speed is of a concern -- or if nItems is 
very large and nFiltered is expected to be much smaller -- it might be 
better to dynamically size the array like this:

Function uArrayFilter4(aArray, bFilter)
LOCAL nItems    := len(aArray)
LOCAL aFiltered := Array(100)
LOCAL nFiltered := 0
LOCAL nItem     := 0
   for nItem := 1 to nItems
      if Eval(bFilter, aArray[nItem])
         if (++nFiltered % 100) == 0
            ASize(aFiltered, 100 + nFiltered)
         endif
         aFiltered[nFiltered] := aArray[nItem]
      endif
   next nItem
   ASize(aFiltered, nFiltered)
return (aFiltered)

Hope that helps,

Andreas

Andreas Gehrs-Pahl
Absolute Software, LLC

phone: (989) 723-9927
email: Andreas@AbsoluteSoftwareLLC.com
web:   http://www.AbsoluteSoftwareLLC.com
[L]:   https://www.LinkedIn.com/in/AndreasGehrsPahl
[F]:   https://www.FaceBook.com/AbsoluteSoftwareLLC

Jonathan,

Even better is this code, as it removes the modulus calculation from the 
loop which should make it even faster than 3 and 4:

Function uArrayFilter5(aArray, bFilter)
LOCAL nItems    := len(aArray)
LOCAL nMaxItems := 100
LOCAL aFiltered := Array(nMaxItems)
LOCAL nFiltered := 0
LOCAL nItem     := 0
   for nItem := 1 to nItems
      if Eval(bFilter, aArray[nItem])
         if ++nFiltered > nMaxItems
            ASize(aFiltered, nMaxItems += 100)
         endif
         aFiltered[nFiltered] := aArray[nItem]
      endif
   next nItem
   ASize(aFiltered, nFiltered)
return (aFiltered)

Hope that helps,

Andreas

Andreas Gehrs-Pahl
Absolute Software, LLC

phone: (989) 723-9927
email: Andreas@AbsoluteSoftwareLLC.com
web:   http://www.AbsoluteSoftwareLLC.com
[L]:   https://www.LinkedIn.com/in/AndreasGehrsPahl
[F]:   https://www.FaceBook.com/AbsoluteSoftwareLLC

Jonathan,

>I'm still surprised that the AEVAL method was slower but perhaps somehow 
>in the FOR / NEXT process the bFilter works more efficiently???

The reason for this is simply that specialized code usually runs faster than 
generic code. The AEval() function internally  isn't much different than a 
for/next loop, but it is written more generically and probably needs to call 
more sub-routines. So instead of several explicit lines of code, you use a 
single function. Less code to write, but actually more code to execute.

If you feel the need for speed, it usually helps to write optimized code. If 
you don't need that last little bit of speed (or memory, etc.) optimization, 
you can use the generic, high-level code and you might save programming -- 
rather than execution -- time, as well as some lines of source code.

For time-sensitive code, though, it is always better to write specific, 
optimized, code, that only does what you want it to do, rather than using 
multi-purpose, generic code, which can handle more possible scenarios, but 
runs a little slower.

Hope that explains that.

Andreas

Andreas Gehrs-Pahl
Absolute Software, LLC

phone: (989) 723-9927
email: Andreas@AbsoluteSoftwareLLC.com
web:   http://www.AbsoluteSoftwareLLC.com
[L]:   https://www.LinkedIn.com/in/AndreasGehrsPahl
[F]:   https://www.FaceBook.com/AbsoluteSoftwareLLC

On 8/6/2020 7:01 PM, Andreas Gehrs-Pahl wrote:
> Jonathan,
> 
>> I'm still surprised that the AEVAL method was slower but perhaps somehow
>> in the FOR / NEXT process the bFilter works more efficiently???
> 
> The reason for this is simply that specialized code usually runs faster than
> generic code. The AEval() function internally  isn't much different than a
> for/next loop, but it is written more generically and probably needs to call
> more sub-routines. So instead of several explicit lines of code, you use a
> single function. Less code to write, but actually more code to execute.
> 
> If you feel the need for speed, it usually helps to write optimized code. If
> you don't need that last little bit of speed (or memory, etc.) optimization,
> you can use the generic, high-level code and you might save programming --
> rather than execution -- time, as well as some lines of source code.
> 
> For time-sensitive code, though, it is always better to write specific,
> optimized, code, that only does what you want it to do, rather than using
> multi-purpose, generic code, which can handle more possible scenarios, but
> runs a little slower.
> 
> Hope that explains that.
> 
> Andreas
> 
Hi Andreas,

When I was playing with my code and subsequently posting it your name 
popped up in my mind... half expecting (OK 90%) you to have a better 
solution... As this was just an experiment... primarily focused on AEVAL 
vs FOR/NEXT with your contribution I learned more than I expected!

For fun I will take my existing code... adjust as suggested and do 
additional timings.

Thanks for contributing to my ongoing education... sincerely appreciated

Stay well and safe... regards... Jonathan

jonathan.leeming@familycentre.org
Edmonton, Alberta, Canada

On 06/08/20 20:01, Andreas Gehrs-Pahl wrote:
> Jonathan,
> 
>> I'm still surprised that the AEVAL method was slower but perhaps somehow
>> in the FOR / NEXT process the bFilter works more efficiently???
> 
> The reason for this is simply that specialized code usually runs faster than
> generic code. The AEval() function internally  isn't much different than a
> for/next loop, but it is written more generically and probably needs to call
> more sub-routines. So instead of several explicit lines of code, you use a
> single function. Less code to write, but actually more code to execute.
> 
> If you feel the need for speed, it usually helps to write optimized code. If
> you don't need that last little bit of speed (or memory, etc.) optimization,
> you can use the generic, high-level code and you might save programming --
> rather than execution -- time, as well as some lines of source code.
> 
> For time-sensitive code, though, it is always better to write specific,
> optimized, code, that only does what you want it to do, rather than using
> multi-purpose, generic code, which can handle more possible scenarios, but
> runs a little slower.
> 
> Hope that explains that.
> 
> Andreas
> 

Gracias Andreas for share you experience.

Best Regards
Osvaldo Ramirez

Yes. It worked. 

The arrays we are using for this purpose are small. So no performance
issues.

Kiron

On Thu, 06 Aug 2020 14:26:26 +0200, Pascal Boivin wrote:

>You have to make a copy of the array. Code below not tested!
>
>aNewArray := FilterArray(aArray1000, {|x| "KIRON" $ x} )
>
>FUNCTION FilterArray(aOriginal, cbFilter)
>LOCAL aReturn := {}, I
>
>  For I := 1 To Len(aOriginal)
>    IF Eval(cbFilter, aOriginal[I])
>      aAdd(aReturn, aOriginal[I])
>    ENDIF
>  NEXT
>RETURN aReturn