Some keynotes about Areas->Boundary Checking->Edge mode

Why is Edge mode only supported in certain render engines?​

Edge mode works at render time, removing elements which are outside the scatter area. Therefore, it is required to implement a custom shader with certain capabilities, but actually only V-Ray Allows this technique. The "trimmed part" is treated by the V-Ray render engine as a kind of "Matte object" internally (this way render engine ignores it for its calculations).

For other supported renderers, you will need to add a Forest Edge map to the material's Opacity input. Forest Material Optimiser can do this for you automatically.

What do I have to do to preview edge mode in the viewport?​

Edge Mode does not preview in viewports by default - to improve viewport performance edge trimming is calculated only at render time. This means the effects of this mode will not be previewed in the viewports. However, previews can be enabled by holding down the left CTRL key and clicking on the Edge radio button. (Points-Cloud Viewport mode only)

AO pass doesn't work properly with the Edge mode?​

When rendering Ambient Occlusion is an extra pass, to get the VRayDirt map used in VRayExtraTex render Element to work properly, please mark the work with transparency option under VRayDirt Parameters.

Why I'm getting some render artefacts while using the Edge mode? What should I do to avoid it?​

In the case of very high density scatters (like dense grass), many overlapping objects could be generated and V-Ray may exceed its default transparency limit (which can cause render artefacts). To fix it:

1. Use lower density values, increasing Distribution Map.> Density X/Y Size. (in many cases this also will improve the render performance, because there are fewer items to render)
2. Under V-Ray render settings, increase Global switches.> Max transp. levels(default value is 50). Sometimes may be necessary to modify Secondary rays bias also.