Workaround for using 'validate.required' on form file fields

A long standing issue of the Form Plugin is that the attribute validate.required on file fields prevents the form to be submitted.
I have created a small plugin which circumvents this problem.

Instead of using validate a new attribute restrictions is added like this:

images:
    label: 'Upload images (JPG, PNG) (at least 2)'
    type: file
    destination: user/data/files
    button_text: Upload images
    multiple: true
    accept:
        - image/jpeg
        - image/png
    restrictions:
      required: true
      minimum: 2

Likewise a maximum number of files to be uploaded can be set using maximum.

If the responses to this post show that this plugin has added value I could publish it for all to use. Please let me know.

In any case I would appreciate help in positioning and styling the error message.

3 Likes

Without having tried this yet, thank you so much for this. Such great news!!

When I have a chance, I will deploy it for one client in particular. Have wasted so much time trying to work around this and bumping the issue on Github, as well as being mildly rebuked by the client because submissions got through without attachments. You rock!

@bleutzinn,

In any case I would appreciate help in positioning and styling the error message.

Without creating a repo on GitHub, it would be very hard for others to help you… :slight_smile:

And @hughbris won’t be able to test it…

1 Like

Point taken :+1:
Working on it!

1 Like

Is everybody ready for a test drive?

Instead of creating a separate plugin I have integrated the workaround in an already existing form focused plugin: the Form Prefiller Plugin.

To test the workaround download and install a modified plugin version from the special ‘dev’ branch on GitHub.
The ReadMe file has been updated so use that to learn how to replace(!) the attribute validate.required by the new attribute restrictions.required.

Please report your findings and the like here and post issues in the GitHub repo.

My experience in creating and maintaining plugins has often felt quite solitary. I see this as an opportunity to spark discussion and collaboration. Let’s see whether or not we can further enhance this plugin, both regarding it’s value and usability. Along the way we might even agree upon some best practices which will improve all plugins in the future.
To that end I have a couple of points we might discuss:

  1. Do we want more validation on the client side like on mime types and maximum filesize?

  2. The PHP library ‘simple_html_dom’ is not installed via composer but added inside the vendor folder and included using an include_once statement. Is it desirable to let composer handle the install so it autoloads and a separate include_once isn’t needed? If so, how?

  3. Every now and then users report conflicting plugins. One way to mitigate conflicts is to use namespaces in a plugin’s Javascript and CSS code. I intend to add these upcoming weekend unless this is regarded as overkill or maybe there’s a better way.

  4. At this moment the language translation is generic in the sense that it ‘talks’ about ‘files’. But what about the use case of for example a file field to upload a PDF document and one to upload one or more images. In such a case proper label texts should indicate the kind or type of file like “document” or “PDF” and “image(s)”. Using the file field name to this purpose does not do it. So in this case another attribute would be needed?

  5. Last but not least I’m actually looking for a method or guideline to keep the languages.yaml file save and outside this plugin’s folder so it does not get overwritten when updating the plugin. It is pretty much the same as with templates but as far as I know there is no equivalent for onTwigTemplatePaths to set alternative locations to look for language files. Or is there?

Happy test driving! I’m curious to see how far we’ll go from here!

1 Like

I added a namespace (“gravFormPrefillerPlugin”) to the Javascript code according to the “revealing module pattern” as described in JavaScript best practices - W3C Wiki
That article dates from 2015 and maybe there are better methods today but it’s a way which I understand and could implement.

Something I forgot to mention is that the error messaging logic now is inside the Javascript. Regarding customizability it would be much better to move that logic into Twig. Suggestions or a PR to that end is much appreciated.

Finally as I pointed out in the original post is the matter of positioning and styling error messages next to the file field. Again help is welcomed.

@bleutzinn,

I would prefer two separate plugins. One for filling form fields and another one for fixing the ‘required’ issue of the File field. I think the two functionalities are too different to combine in one plugin.

I’ve played around a bit and created two versions with a possible fix of the File issue.

  1. One with the default error handling of the Form plugin.
  2. One using Twig for error handling.

Ad 1. Default error handling

  • Create form user/pages/03.uploads/form.md, containing

    ---
    title: Uploads
    form:
      name: fixformfield
      fields:
        name:
          type: text
        uploads:
          type: file
          label: 'Upload <span class="required"> *</span'
          multiple: true
          restrictions:
            required: true
            minimum: 2
            maximum: 2
      buttons:
        submit:
          value: submit
      process:
        message: Thank you!
    ---
    
  • Create plugin fixfilefield using $ bin/plugin devtools new-plugin.

  • Replace the content of fixfilefield.php with:

    Content of fixfilefield.php
    <?php
    
    namespace Grav\Plugin;
    
    use Composer\Autoload\ClassLoader;
    use Grav\Common\Data\ValidationException;
    use Grav\Common\Form\FormFlash;
    use Grav\Common\Plugin;
    use Grav\Plugin\Form\Form;
    use RocketTheme\Toolbox\Event\Event;
    
    /**
     * Class FixfilefieldPlugin
     * @package Grav\Plugin
     */
    class FixfilefieldPlugin extends Plugin
    {
      /**
       * @return array
       *
       * The getSubscribedEvents() gives the core a list of events
       *     that the plugin wants to listen to. The key of each
       *     array section is the event that the plugin listens to
       *     and the value (in the form of an array) contains the
       *     callable (or function) as well as the priority. The
       *     higher the number the higher the priority.
       */
      public static function getSubscribedEvents(): array
      {
        return [
          'onPluginsInitialized' => [
            // Uncomment following line when plugin requires Grav < 1.7
            // ['autoload', 100000],
            ['onPluginsInitialized', 0]
          ]
        ];
      }
    
      /**
       * Composer autoload
       *
       * @return ClassLoader
       */
      public function autoload(): ClassLoader
      {
        return require __DIR__ . '/vendor/autoload.php';
      }
    
      /**
       * Initialize the plugin
       */
      public function onPluginsInitialized(): void
      {
        // Don't proceed if we are in the admin plugin
        if ($this->isAdmin()) {
          return;
        }
    
        // Enable the main events we are interested in
        $this->enable([
          // Put your main events here
          'onFormValidationProcessed' => ['onFormValidationProcessed', 0],
        ]);
      }
    
      public function onFormValidationProcessed(Event $event)
      {
        /** @var Form */
        $form = $event['form'];
        $fields = $form->getFields();
    
        $messages = [];
        /** @var FormFlash */
        $flash = null;
    
        foreach ($fields as $key => $field) {
          if ($field['type'] === 'file' && isset($field['restrictions'])) {
            $flash = $form->getFlash();
            $uploads = $flash->getFilesByField($key);
    
            if ($field['restrictions']['required'] === true && count($uploads) === 0) {
              $messages[$key][] = "Field '{$field['name']}' is required";
            }
    
            if (isset($field['restrictions']['minimum']) && count($uploads) < $field['restrictions']['minimum']) {
              $messages[$key][] = "You must upload a minimum of {$field['restrictions']['minimum']} files.";
            }
    
            if (isset($field['restrictions']['maximum']) && count($uploads) > $field['restrictions']['maximum']) {
              $messages[$key][] = "You are allowed to upload {$field['restrictions']['minimum']} files.";
            }
          }
        }
    
        if ($flash !== null) {
          $flash->delete();
        }
    
        if (count($messages) > 0) {
          $exception = new ValidationException('Some fields have incorrect or missing input.');
          $exception->setMessages($messages);
    
          throw $exception;
        }
      }
    }
    
  • Screenshot of incorrect input:

Ad 2. Using Twig for error handling.

  • Same form definition as above

  • Create plugin fixfilefield using $ bin/plugin devtools new-plugin.

  • Replace the content of fixfilefield.php with:

    Content of fixfilefield.php
    <?php
    
    namespace Grav\Plugin;
    
    use Composer\Autoload\ClassLoader;
    use Grav\Common\Data\ValidationException;
    use Grav\Common\Form\FormFlash;
    use Grav\Common\Plugin;
    use Grav\Plugin\Form\Form;
    use RocketTheme\Toolbox\Event\Event;
    
    /**
     * Class FixfilefieldPlugin
     * @package Grav\Plugin
     */
    class FixfilefieldPlugin extends Plugin
    {
      /**
       * @return array
       *
       * The getSubscribedEvents() gives the core a list of events
       *     that the plugin wants to listen to. The key of each
       *     array section is the event that the plugin listens to
       *     and the value (in the form of an array) contains the
       *     callable (or function) as well as the priority. The
       *     higher the number the higher the priority.
       */
      public static function getSubscribedEvents(): array
      {
        return [
          'onPluginsInitialized' => [
            // Uncomment following line when plugin requires Grav < 1.7
            // ['autoload', 100000],
            ['onPluginsInitialized', 0]
          ]
        ];
      }
    
      /**
       * Composer autoload
       *
       * @return ClassLoader
       */
      public function autoload(): ClassLoader
      {
        return require __DIR__ . '/vendor/autoload.php';
      }
    
      /**
       * Initialize the plugin
       */
      public function onPluginsInitialized(): void
      {
        // Don't proceed if we are in the admin plugin
        if ($this->isAdmin()) {
          return;
        }
    
        // Enable the main events we are interested in
        $this->enable([
          // Put your main events here
          'onTwigTemplatePaths' => ['onTwigTemplatePaths', 10],
          'onTwigLoader' => ['onTwigLoader', 0],
          'onFormInitialized' => ['onFormInitialized', 0],
          'onFormValidationProcessed' => ['onFormValidationProcessed', 0],
        ]);
      }
    
      public function onTwigTemplatePaths()
      {
        $this->grav['twig']->twig_paths[] = __DIR__ . '/templates';
      }
    
      public function onTwigLoader()
      {
        $form_path = $this->grav['locator']->findResource('plugins://form') . DIRECTORY_SEPARATOR . 'templates';
        $this->grav['twig']->addPath($form_path, 'formplugin');
      }
    
      public function onFormInitialized(Event $event)
      {
        $form = $event['form'];
        $fields = $form->getFields();
    
        foreach ($fields as $field) {
          if ($field['type'] === 'file' && isset($field['restrictions'])) {
            $this->grav['twig']->twig_vars['fixfilefield'] = [
              'restrictions' => $field['restrictions']
            ];
          }
        }
      }
    
      public function onFormValidationProcessed(Event $event)
      {
        /** @var Form */
        $form = $event['form'];
        $fields = $form->getFields();
    
        $messages = [];
        /** @var FormFlash */
        $flash = null;
    
        foreach ($fields as $key => $field) {
          if ($field['type'] === 'file' && isset($field['restrictions'])) {
            $flash = $form->getFlash();
            $uploads = $flash->getFilesByField($key);
    
            if ($field['restrictions']['required'] === true && count($uploads) === 0) {
              $messages[$key][] = "Field is required";
            }
    
            if (isset($field['restrictions']['minimum']) && count($uploads) < $field['restrictions']['minimum']) {
              $messages[$key][] = "You must upload a minimum of {$field['restrictions']['minimum']} files.";
            }
    
            if (isset($field['restrictions']['maximum']) && count($uploads) > $field['restrictions']['maximum']) {
              $messages[$key][] = "You may upload a maximum of {$field['restrictions']['minimum']} files.";
            }
          }
        }
    
        if ($flash !== null) {
          $flash->delete();
        }
    
        $mergedVars = array_merge($this->grav['twig']->twig_vars['fixfilefield'], [
          'errors' => $messages,
        ]);
    
        $this->grav['twig']->twig_vars['fixfilefield'] = $mergedVars;
    
        if (count($messages) > 0) {
          throw new ValidationException('Some fields have an incorrect or missing value.');
        }
      }
    }
    
  • Create the following folder structure inside the plugin:

    user/plugins/fixfilefield/templates
    └── forms
      ├── field.html.twig
      └── form.html.twig
    

    Add the following to field.html.twig:

    Content of field.html.twig
    {% extends "@formplugin/forms/field.html.twig" %}
    
    {% block label %}
      {% if field.type == 'file' %}
        {% set form_field_required = fixfilefield.restrictions.required %}
      {% endif %}
    
      {{ parent() }}
    {% endblock %}
    

    This will add the red asterix (*) to the label.

  • Add the following to form.html.twig:

    Content of form.html.twig
    {% extends "@formplugin/forms/form.html.twig" %}
    
    {% block inner_markup_field_close %}
      {% for error in fixfilefield.errors[field_name] %}
        {# Yes styling should be in a css file #}
        <div style="color:red;font-size:.6rem;font-weight:normal">{{ error }}</div>
      {% endfor %}
    {% endblock %}
    

    This will add the error messages below the file field

  • Screenshot of incorrect input:

Notes:

  • Yes it’s not perfect… It’s a proof of concept.
  • There is room for improvements.
  • Haven’t figured out yet how to keep the selected files inside the File field after an error has happened.

I’m working on a separate Fix File Field Plugin as @pamtbaau suggested and among other things basing that on the field.html.twig template shown in the previous post. So this question is for the Twig wizards amongst you.

Grav allows you to add custom data- attributes to the HTML of a form field; read about it at the bottom of the section Common Field Attributes in the docs.

I want to use that method to pass a minimum number of files which should be uploaded and use it’s value to do a client side validation.
BTW setting a maximum number of files is supported by the form out of the box by setting the limit attribute.

As a test I set this field definition:

images:
    label: ' Images upload'  
    type: file
    datasets:
        formkey: formvalue
    multiple: true
    accept:
        - image/jpeg
        - image/png

In the browser developer tools I can see the attribute data-formkey has been added to the hidden input field (BTW this field is hidden through CSS):
<input type="file" multiple="multiple" accept="image/jpeg,image/png" class="form-input" data-formkey="formvalue">

Then again as a test I want to add another key value pair to the datasets attribute. For that I modified the field.html.twig template like so:

{% extends "@formplugin/forms/field.html.twig" %}

{% block label %}

  {% if field.type == 'file' %}
    {{ dump(field) }}
    {% set form_field_required = field.restrictions.required %}

    {% set myNewKeyValue = {
        'newKey': 'newValue'
    } %}

    {% set field = field|merge({ 'datasets': (field.datasets|default([]))|merge(myNewKeyValue) }) %}
    {{ dump(field) }}

  {% endif %}

  {{ parent() }}
{% endblock %}

Looking at the Debug Bar the second dump(field) shows:

"datasets" => array:2 [
  "formkey" => "formvalue"
  "newKey" => "newValue"
]

So inside the field.html.twig template the extra key value pair has successfully been added to the datasets attribute or array.

The problematic part is that this modified version of datasets does not get transferred to the HTML form. The hidden input field still has the data-formkey attribute only:
<input type="file" multiple="multiple" accept="image/jpeg,image/png" class="form-input" data-formkey="formvalue">. The newKey attribute is lost somewhere.

So the question is where and how does this go wrong and what is the correct way of adding data- attributes via Twig?

@bleutzinn,

Three alternatives:

1. Using Twig

When searching for datasets in the template folder of the Form plugin, I find a reference inside user/plugins/form/templates/forms/default/field.html.twig which is located inside block {% block input_attributes %}.

When adding the following inside /user/plugins/fixfilefield/templates/forms/field.html.twig the attributes will be set correctly:

{% block input_attributes %}
  {% if field.type == 'file' %}
    {# Simple way of setting attributes #}
    xxx="yyy"

    {# Using the field.datasets property #}
    {% set myNewKeyValue = {
        'twigDataKey': 'twigDataValue'
      } %}
    {% set field = field|merge({ 'datasets': (field.datasets|default([]))|merge(myNewKeyValue) }) %}
  {% endif %}

  {{ parent() }}
{% endblock %}

When using the following field definition inside page form.md:

...
fields:
  uploads:
    type: file
    datasets:
      formDefDataKey: formDefDataValue

The resulting HTML will be:

<input type="file" accept="image/*" class="form-input " 
  xxx="yyy"                                 <-- from Twig
  data-formdefdatakey="formDefDataValue"    <-- from form definition
  data-twigdatakey="twigDataValue"          <-- from Twig
>

2. Using PHP to add field attributes:

Alternatively, without using any Twig you could simply use PHP:

public function onFormInitialized(Event $event)
{
  $form = $event['form'];
  $fields = $form->getFields();

  foreach ($fields as $key => $field) {
    if ($field['type'] === 'file') {
      $fields[$key]['datasets']['phpKey'] = 'phpValue';
    }
  }

  $form->setFields($fields);
}

Without any template file, this will generate (using the same form definition as above):

<input type="file" accept="image/*" class="form-input " 
  data-formdefdatakey="formDefDataValue"  <-- from form definition
  data-phpkey="phpValue"                  <-- From PHP
>

3. Using PHP to injecting object into HTML:
Since all you want is to pass meta data about the field into HTML to allow js to validate the form, you could also inject a PHP object into HTML using Asset Manager using inlineJs(). Js can access the object and use it to validate the form.

I don’t really see the necessity to inject the meta data into the HTML <input> field itself. It only adds complexity: More complexity to inject and more the retrieve from DOM.

@pamtbaau again thanks for your answers. I guess the secret in why the Twig way didn’t work for me and did for you is {% block input_attributes %} and I especially like the “Simple way of setting attributes”. I wonder how you found out about this and learn about how to do thing in general.

It’s typical of Grav imho to see there are many ways leading to Rome so to say. Thanks for showing these three.

I am aware of the added complexity and the possible relative high cost in terms of performance when adding data as element attributes like this. For me the data attribute remains the most appropriate place to store properties of the element in question. The ‘PHP way’ is a very good alternative too. I think I’ll take that route.

1 Like

The problem is that variables in Twig are only accessible in their scope. Tags like {% block %} , {% for %} , {% nav %} create a ‘scope’, which means that variables defined inside them, can’t be accessed from outside. Therefor you need to use the correct ‘block’ to define the variables.

  1. I just remembered an earlier topic on extending template and its solution added to the docs (thanks @Karmalakas): Extend base template of inherited theme.
  2. By having a good look at the generated HTML code of the file field.
  3. By searching the code using unique terms found in the file field.
  4. By combining 1. and 3. with some trial and error.
  1. By being curious about what the answer to an interesting post on the forum might be…
  2. By cruising the docs, the API and code quite a bit while experimenting to solve 1.

Search the web about simple vs complex solutions. Most answers will be like:

Choose the simple one. It meets spec, it’s easier to understand, it’s easier to maintain, and it’s probably a whole lot less buggy.

1 Like

@bleutzinn Sorry I haven’t got around to trying this out yet. From my comment, you may have thought I would do so quite soon. I’m keen to, but it’s further down my list of priorities. (I guess if that client hassles me again about submissions without attachments, I will jump into it.)

I did originally take a look at your solution and was put off trying it when I saw it was bundled into another plugin, so @pamtbaau was right to point that out and you did the right thing changing it :slight_smile:

@hughbris No need to apologize. Sometimes “life gets in the way” so to say. In the mean time I have been “reworking the workaround” and feel it is time for an update on that.

It will be named the “FixFileField Plugin”, thanks to @pamtbaau for the name and urging me to make a separate plugin. It will be not because I’m totally convinced but because I want to create things that are useful AND get used and not put aside for some reason.

When reworking I realized that though I had client side validation in place for the newly introduced attributes restrictions.required, the restrictions.minNumberOfFiles and the restrictions.maxNumberOfFiles the server side validation was missing. That is because the workaround only addresses problems in the Form Plugin frontend code (Javascript). Since these new attributes don’t exist as far as the Form Plugin is concerned it can’t validate on them.

Then I thought of allowing the use of the standard validation.required and the limit file field attributes. Since the limit attribute sets an upper limit to the number of files allowed to be uploaded I felt I needed another one for setting a lower limit, thus appropriately called lower_limit.

Obviously there will be no server side check whether or not the number of uploaded files is equal to or exceeds the setting of lower_limit as that setting is unknown to the Form Plugin but I don’t think that will pose a problem.

So this change is closer to the default behaviour and closer to existing user experience and contributes to a more unobtrusive solution. However in that regard I wasn’t happy with some user experience quirks. Mainly when uploading more files than limit allows.
For instance when limit is set to 1 the file picker allows selecting a single file only. Which is good. But once that file has appeared in the file field as a thumbnail the user can again click that field and select yet another file or drop another file into the field. This is bad behaviour. The Form plugin signals the illegal extra uploaded files by putting a cross symbol on the thumbnail but still. The user shouldn’t have gotten the opportunity to do something which isn’t allowed. So I try to fix or work around that too.

And so a seamingly and relatively simple workaround got into a project of it’s own. It is a typical example of the 20-80 rule. It took me 20% of the time to get to 80% of the functionality but the remaining 20% is taking 80% more time or even more.
Besides that I find it hard to draw the line somewhere. The only reason I sometimes get things done is persistence and thus not throwing the towel.

However, I can’t estimate when or even if I will succeed in fixing the UX problems I described. And at the same time I got the impression the plugin could be useful even with the UX problems. Also those exist as long as the Form Plugin is around and as far as I know no one complained about this misbehaviour too.

That being said I think I should wrap things up, create a test version and put a couple of issues aside as “to do”. When, I can’t really tell. Hopefully by the end of this week.

I’ll keep you all posted.

1 Like

For everyone willing to take a peek and beyond, a demo is currently online at https://rwgc.nl/lab/demo-grav-plugin-fixfilefield/

With ‘beyond’ I mean diving into the code and especially the Javascript where the validation takes place.

The plugin may be downloaded from there or from GitHub and the full form page source is shown on that site as well.

This ‘alpha’ version still has some issues which I think I can’t fix myself. So any help is more than welcome and badly needed to make it into a proper workaround. A description of these issues is included on a separate page.

Thank you in advance, and I’m looking forward to your findings and, ideally, assistance.